Merge pull request #5 from intelops/cueval

CUE validation/generation, add cli flags, templates, and update README

CONTRIBUTION.md

@@ -92,4 +92,37 @@ git checkout <your_branch_name>
 git merge main
 ```
 
+## Contributing by adding a Cue schema to the project
+
+Genval leverages upstream Kubernetes and CRD APIs for validation and configuration generation. For technologies not currently supported, users can extend Genval's capabilities by adding Cue schemas.
+
+The workflow for adding a Cue schema is as follows:
+
+- Clone the repository by following the steps outlined [above](#set-up-your-local-development-environment) and navigate to the `genval` directory.
+- Identify the technology and the package URL that exposes the APIs you want to add schemas for. In the following example, we'll import schemas for [TektonCD](https://tekton.dev/docs/):
+
+```shell
+$ go get github.com/tektoncd/pipeline/pkg/apis/...
+# This downloads the Go types for the API.
+$ cue get go github.com/tektoncd/pipeline/pkg/apis/...
+# This imports the Go types into Cue schemas for TektonCD within the 'cue.mod' directory.
+```
+Once the APIs are imported into Cue, you can reference them in your definitions for TektonCD, similar to import statements in Golang:
+
+```shell
+package tekton
+
+import "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1"
+
+#Pipeline: v1beta1.#Pipeline & { 
+	apiVersion: string | *"tekton.dev/v1beta1"
+	kind:       string | *"Pipeline"
+    ...
+
+```
+
+Now, you can easily reference the Cue schemas for #Pipeline by utilizing the imported schemas as v1beta1.#Pipeline.
+
+
+
 ## All the best! 🥇

Makefile

@@ -40,4 +40,4 @@ lint: ## Run a linter on the codebase using golangci-lint.
 
 
 build: ## builds the GenVal app for defined OS/Arch by passing GOOS=$(GOOS) GOARCH=$GOARCH args.| Example usage `make build GOOS=linux GOARCH=amd64`
-	@GOOS=$(GOOS) GOARCH=$(GOARCH) go build -ldflags="-X main.Version=$(shell git describe --tags --abbrev=0)" -o ./bin/genval
+	@GOOS=$(GOOS) GOARCH=$(GOARCH) go build -ldflags="-X main.Version=$(shell git describe --tags --abbrev=0)" -o ./bin/genval ./cmd

README.md

@@ -1,122 +1,209 @@
-[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/7843/badge)](https://www.bestpractices.dev/projects/7843)
+# Genval: Simplifying Configuration Management
 
-# genval
-Verify, Generate and Validate dockerfile, K8s Manifests, Helm, Timoni, Kustomize, K8s Operators, Tekton, GitOps, K8s Infa YAML files, etc. programmatically.
+
 
+Genval is a versatile Go utility that simplifies configuration management for a wide range of tools, including Dockerfile, Kubernetes manifests, Helm, Timoni, Kustomize, Kubernetes Operators, Tekton, GitOps, Kubernetes Infrastructure YAML files, and more.
 
-## Simplifying Dockerfile Validation and Generation
+  
 
-**GenVal** is a powerful utility written in Golang that simplifies the process of Dockerfile validation and generation. Whether you're new to Docker or a seasoned pro, GenVal can help you create Dockerfiles that adhere to best practices for security and optimization.
+## Streamlining validation and Generation of Configurations
 
-## Why GenVal?
+  
 
-Managing Dockerfiles can be complex. It's crucial to ensure they follow best practices, but this can be time-consuming and error-prone. GenVal takes care of the heavy lifting by automating these processes, making Dockerfile management more efficient.
+**Genval** is a robust utility written in Golang that streamlines the management of configurations for various tools. Whether you need Dockerfiles, YAML/JSON manifests for Kubernetes, or custom resource definitions (CRDs), Genval simplifies the process of validation and generation for multiple configuration artifacts.
 
-## Features
+  
 
-- **Input Validation**: GenVal begins by validating your input, which can be provided in JSON format. This initial step ensures that your input data is correctly structured.
+## Why Genval?
 
-- **Dockerfile Generation**: Once your input passes validation, GenVal generates a Dockerfile for you based on the provided data. This Dockerfile will be tailored to your specifications.
+  
 
-- **Best Practice Validation**: After generating the Dockerfile, GenVal doesn't stop there. It checks your Dockerfile against a set of predefined best practices for security and optimization. 
+Managing configurations across different tools can be a daunting task. Ensuring that these configurations adhere to best practices is critical but can be time-consuming and error-prone. Genval automates these processes, making configuration management more efficient.
 
-- **Informative Feedback**: If GenVal detects any issues or discrepancies during the best practice validation, it provides informative feedback. This feedback helps you understand what needs improvement in your Dockerfile.
+  
 
+## Key Features
 
-> Note: Currently, `genval` requires a pre-defined structure for the input json to be provided for generating and validating the generated Dockerfile. The sample input `json` files are stored in `/templates/dockerFile-samples` directory of the project.
+
+
+### Dockerfile Management
+
+
+
+-  **Input Validation**: Genval validates your input, typically provided in JSON format, to ensure correct structure.
+
+
+
+-  **Dockerfile Generation**: After successful validation, Genval generates a Dockerfile based on your input, tailored to your specifications.
+
+
+
+-  **Best Practice Validation**: Genval doesn't stop at generation; it checks your Dockerfile against predefined best practices for security and optimization.
+
+
+
+-  **Informative Feedback**: If Genval detects issues during best practice validation, it provides informative feedback to help you improve your Dockerfile.
+
+
+
+> Note: For Dockerfile validation and generation, `genval` expects a predefined structure for the `JSON` file provided to the `--value` flag. Sample `.json` files can be found in the `./templates/dockerFile-samples` directory.
+
+
+
+### Managing Kubernetes and CRD Manifests
+
+
+
+- Genval validates input for required structure based on the tool in use. It can enforce best practices while authoring configuration files for tools like Kubernetes and various CRDs.
+
+
+
+- Users can provide minimal and custom configurations for a given resource, with Genval populating all necessary fields based on security best practices recommended by the community.
+
+
+
+- Genval generates YAML manifests according to the required format for the specified resource and tool.
+
+
 
 ## Getting Started
 
-Using GenVal is straight forward:
+  
 
-`genval` is available as a binary executable for all major platforms, the binaries can be downloaded from the official [release page](https://github.com/intelops/genval/releases)
+To use Genval:
 
-## Verifying the binary signatures 
+  
 
-The GenVal release process signs the binaries using Cosign's keyless signing mode. To verify a perticular binary, retrive the release checksum together with its signature and public certificate for the desired `TAG`.
+- Download the `genval` binary for your platform from the official [release page](https://github.com/intelops/genval/releases).
 
-> Note: Cosign 2.0 requires the `--certificate-identity` and `--certificate-oidc-issuer` options.
-For more information please refer to the [Sigstore blog](https://blog.sigstore.dev/cosign-2-0-released/). 
+
 
-```shell
-wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt
-wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.pem
-wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.sig
-# Remember to update the tag (v0.0.1) in the above url for your desired release
-```
-Verify the downloaded blob i.e the `checksums.txt` using cosign tool
+## Verifying Binary Signatures
+
+  
+
+Genval's release process signs binaries using Cosign's keyless signing mode. To verify a specific binary, retrieve the release checksum, signature, and public certificate for your desired `TAG`. Detailed instructions are available in the [Sigstore blog](https://blog.sigstore.dev/cosign-2-0-released/).
+
+  
 
 ```shell
-cosign verify-blob \
-    --certificate-identity "https://github.com/intelops/genval/.github/workflows/release.yaml@refs/tags/v0.0.1" \
-    --certificate-oidc-issuer "https://token.actions.githubusercontent.com" \
-    --cert ./checksums.txt.pem \
-    --signature ./checksums.txt.sig \
-    ./checksums.txt
-```
 
-If the verification is successful, we would ge an output `Verified OK`.
+# Example commands to verify a binary
 
+$ wget  https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt
+$ wget  https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.pem
+$ wget  https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.sig
 
-**Installation**: To install GenVal on your system. You can downlod the executable binary from the releases page, make it an executible and you are ready to go.
+  
 
-```shell
-curl https://github.com/intelops/genval/releases/download/v0.0.1/genval_0.0.1_<os/arch>.tar.gz
-tar -xvf genval_0.0.2_<os/arch>.tar.gz
-chmod +x genval
-# move the executable to PATH
-mv ./genval /usr/local/bin
+cosign  verify-blob  \
+--certificate-identity  "https://github.com/intelops/genval/.github/workflows/release.yaml@refs/tags/v0.0.1"  \
+--certificate-oidc-issuer  "https://token.actions.githubusercontent.com"  \
+--cert  ./checksums.txt.pem  \
+--signature  ./checksums.txt.sig  \
+./checksums.txt
 ```
-For quick start, there are some pre-built templates to build Dockerfile for popular languages in `/templates/dockerFile-sample` directory. 
+If verification is successful, you'll see "**Verified OK.**"
 
-## Build from source
+  
 
-To build `genval` from source, you can follow the instructions as below:
+## Installation
 
-> Note: If you are building from source, please ensure you are building the application from the `main` branch to avoid any unstable behaviour of the tool.  
+
 
+- Download the genval binary from the official [release page](https://github.com/intelops/genval/releases)
 
-```shell
-git clone https://github.com/intelops/genval.git
-```
-Navigate to the project directory
-`cd genval`
+- Move the executable to your `PATH` for convenience.
 
-Now you can build `genval` using the following command:
+
+
+## Quick Start
+
+
+For a quick start, pre-built templates for Dockerfile generation for popular languages can be found in the `./templates/inputs/dockerfile_input` folder.
+
+
+## Building from Source
+
+The easieast way to build the `genval` executable is using the `build` Makefile target.
+`make build`. 
+This will build the binary from source and place the `genval` binary in the `./bin` folder. 
+
+
+
+To build genval from source:
+
+- Clone the Genval repository: `git clone https://github.com/intelops/genval.git`
+
+- Navigate to the project directory: `cd genval`
+
+- Build Genval: `CGO_ENABLED=0 go build -o ./cmd/genval`
+
+
+
+The generated binary, genval, will be available in the current working directory. You can move it to your PATH or use it from the current directory.
+
+Genval offers two modes:
+
+
+-  `container` for Dockerfile validation and generation
+
+-  `cue` for Kubernetes and CRD validation and generation
+
+
+> Both modes accept inputs that include local files and files obtained from a remote URL, such as those from a Git repository. 
+  
+
+### Dockerfile Validation and Generation:
+
+
+Run Genval with the --mode container flag, providing the path to your input JSON or YAML file using the `--reqinput` flag and specifying the desired output path for the generated Dockerfile along with `--inputpolicy` and `--outputpolicy` for validating the input JSON and the generated Dockerfile respectively. Genval will take care of the rest. 
+
+Example:
 
-```shell
-CGO_ENABLED=0 go build -o genval
 ```
-The generated binary will be available in the current working directory, i.e `genval`. You can optionally move the genval executable to your `PATH` or use the relative path `./genval` to use the tool from `pwd`.
+$ genval --mode container --reqinput ./templates/inputs/dockerfile_input/golang_input.json \
+  --output ./Dockerfile-Golang  \
+  --inputpolicy ./templatates/defaultpolicies/rego/inputfile_policies.rego \
+  --outputpolicy ./templatates/defaultpolicies/rego/dockerfile_policies.rego
+ ```
+
+
+> Replace the values provided in the flags with your custom input file and Rego policies.
 
-Now, you are ready to generate your Dockerfile with following command:
+  
 
+**Review Feedback**: Genval provides feedback based on best practice validation. Use this feedback to refine your Dockerfile.
 
-- **Validation and Generation**: Run GenVal, providing the path to your input JSON or YAML file and the desired path for the generated Dockerfile. GenVal will take care of the rest.
 
-`genval ./templates/dockerfile-samples/go-input.json ./Dockerfile-Golang`
+### Validation and Generation of Kubernetes configurations
 
-- **Review Feedback**: GenVal will display feedback based on the best practice validation. If any issues are found, you can use this feedback to refine your Dockerfile.
+
+
+The validation and generation of Kubernetes and CRD manifests are facilitated through the use of [cuelang](https://cuelang.org/docs/). When using Genval for validating and generating Kubernetes and related manifests, make use of the Genval tool in `cue` mode. This mode necessitates JSON input provided via the `--reqinput` flag. Furthermore, you should specify a `resource` flag, indicating the Kubernetes or CRD `Kind` that requires validation. Additionally, attach the `.cue schema definitions` to the `--policy` flag. These policy files can be provided from the users local file system or from a remote URL, like a Git repository.
+
+You have the flexibility to employ multiple `--policy` flags, allowing you to supply distinct `.cue` definitions as needed. For instance, your DevSecOps/Platform engineering team can furnish a schema that enforces security best practices for a specific environment, encompassing all the pertinent mandatory fields. This approach leaves room for custom fields like `metadata`, `image`, `replicas`, specific to a **Deployment**, to be provided by the development teams. In the `cue` mode, development teams can then contribute their customized policies for validation and generation, tailoring the configurations to suit their particular environments.
+
+
+
+Example:
 
-Here's a simple command to use GenVal:
 
 ```shell
-genval <path/to/input-json> <path/to/Dockerfile>
+$ genval  --mode  cue  --reqinput  ./templates/inputs/cue/deploy,json  \
+  --resource  Deployment  \
+  --policy  ./templates/defaultpolicies/cue/deployment.cue  \
+  --policy  ./templates/defaultpolicies/cue/dev-deployment.cue
 ```
 
-Replace `<path/to/input-json>` with the path to your input JSON file and `<path/to/Dockerfile>` with the desired path for the generated Dockerfile.
-
-## Application Flow Diagram: 
-```mermaid
-flowchart
-	2{"Input validation"} -.->|"Yes"| 3(["Generate \nDockerfile"])
-	1["Input JSON file"] --> 2
-	2 -->|"No"| 471222["Update JSON as needed"]
-	793804("Validate \nDockerfile") --> 3
-	3 -->|"No"| 714538["Update JSON as suggested"]
-	3 -->|"Yes"| 4[["Final Dockerfile"]]
-```
+The above command will validate a Deployment manifests using the provided `.cue` schema definitions provided as policies and generate the final YAML manifet in the `cwd`.
+
+> The `--resource` flag in `cue` mode needs a valid Kind, like in above example "Deployment" or StatefulSet, DaemonSet etc.
+
 
+For a detailed workflow illustrating the capabilities of Cue and Genval for validating and generating Kubernetes configurations, you can refer to [this document](./cmd/cueval/example.md).
+The workflow for adding a Cue schema for Kubernetes CRDs is failry easy, and demostrated in the [CONTRIBUTION.md document](./CONTRIBUTION.md/#contributing-by-adding-a-cue-schema-to-the-project).  
 
-GenVal simplifies Dockerfile management by automating validation and generation while ensuring that your Dockerfiles adhere to best practices. It's a valuable tool for both Docker novices and experts.
+### Templates
 
+The `./templates` folder holds some sample files to be used in Genval. the `./templates/inputs` holds JSON input templates for both generating Dockerfiles in `container` mode and Kubernetes manifests in `cue` mode. Similarly, all the default policies for both the modes are stored in `./templates/defaultpolices` directory. User can use these template files to start with and as they go along they can customize these files to suite their specific use cases.