Skip to content

Commit

Permalink
Fix typos, Add Tekton pipeline and ArgoCD schemas, Add example workfl…
Browse files Browse the repository at this point in the history
…ow document and link it in README

Signed-off-by: santoshkal <[email protected]>
  • Loading branch information
santoshkal committed Nov 9, 2023
1 parent 5ff7ef7 commit adc3659
Show file tree
Hide file tree
Showing 8 changed files with 259 additions and 87 deletions.
37 changes: 11 additions & 26 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,28 +90,19 @@ Genval's release process signs binaries using Cosign's keyless signing mode. To

# Example commands to verify a binary

wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt

wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.pem

wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.sig
$ wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt
$ wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.pem
$ wget https://github.com/intelops/genval/releases/download/v0.0.1/checksums.txt.sig



cosign verify-blob \

--certificate-identity "https://github.com/intelops/genval/.github/workflows/release.yaml@refs/tags/v0.0.1" \

--certificate-oidc-issuer "https://token.actions.githubusercontent.com" \

--cert ./checksums.txt.pem \

--signature ./checksums.txt.sig \

./checksums.txt

```

If verification is successful, you'll see "**Verified OK.**"


Expand All @@ -127,11 +118,9 @@ If verification is successful, you'll see "**Verified OK.**"


## Quick Start




For a quick start, pre-built templates for Dockerfile generation for popular languages can be found in the /templates/dockerFile-sample directory

For a quick start, pre-built templates for Dockerfile generation for popular languages can be found in the `./templates/inputs/dockerfile_input` folder.


## Building from Source
Expand All @@ -154,11 +143,8 @@ To build genval from source:

The generated binary, genval, will be available in the current working directory. You can move it to your PATH or use it from the current directory.



Genval offers two modes:



- `container` for Dockerfile validation and generation

Expand All @@ -169,7 +155,6 @@ Genval offers two modes:

### Dockerfile Validation and Generation:



Run Genval with the --mode container flag, providing the path to your input JSON or YAML file using the `--reqinput` flag and specifying the desired output path for the generated Dockerfile along with `--inputpolicy` and `--outputpolicy` for validating the input JSON and the generated Dockerfile respectively. Genval will take care of the rest.
Expand All @@ -188,25 +173,23 @@ $ genval --mode container --reqinput ./templates/inputs/dockerfile_input/golang_


**Review Feedback**: Genal provides feedback based on best practice validation. Use this feedback to refine your Dockerfile.
**Review Feedback**: Genval provides feedback based on best practice validation. Use this feedback to refine your Dockerfile.


### Validation and Generation of Kubernetes configurations



The validation and generation of Kubernetes and CRD manifests are facilitated through the use of [cuelang](https://cuelang.org/docs/). When using Genval for validating and generating Kubernetes and related manifests, make use of the Genval tool in `cue` mode. This mode necessitates JSON input provided via the `--reqinput` flag. Furthermore, you should specify a `resource` flag, indicating the Kubernetes or CRD `Kind` that requires validation. Additionally, attach the `.cue schema definitions` to the `--policy` flag. These policy files can be provided wither store in the users local file system or from a remote URL, like a Git repository.
The validation and generation of Kubernetes and CRD manifests are facilitated through the use of [cuelang](https://cuelang.org/docs/). When using Genval for validating and generating Kubernetes and related manifests, make use of the Genval tool in `cue` mode. This mode necessitates JSON input provided via the `--reqinput` flag. Furthermore, you should specify a `resource` flag, indicating the Kubernetes or CRD `Kind` that requires validation. Additionally, attach the `.cue schema definitions` to the `--policy` flag. These policy files can be provided from the users local file system or from a remote URL, like a Git repository.

You have the flexibility to employ multiple `--policy` flags, allowing you to supply distinct `.cue` definitions as needed. For instance, your DevSecOps team can furnish a schema that enforces security best practices for a specific environment, encompassing all the pertinent mandatory fields. This approach leaves room for custom fields like `metadata`, `image`, `replicas`, specific to a **Deployment**, to be provided by the development teams. In the `cue` mode, development teams can then contribute their customized policies for validation and generation, tailoring the configurations to suit their particular environments.
You have the flexibility to employ multiple `--policy` flags, allowing you to supply distinct `.cue` definitions as needed. For instance, your DevSecOps/Platform engineering team can furnish a schema that enforces security best practices for a specific environment, encompassing all the pertinent mandatory fields. This approach leaves room for custom fields like `metadata`, `image`, `replicas`, specific to a **Deployment**, to be provided by the development teams. In the `cue` mode, development teams can then contribute their customized policies for validation and generation, tailoring the configurations to suit their particular environments.



Example:



```shell

$ genval --mode cue --reqinput ./templates/inputs/cue/deploy,json \
--resource Deployment \
--policy ./templates/defaultpolicies/cue/deployment.cue \
Expand All @@ -218,6 +201,8 @@ The above command will validate a Deployment manifests using the provided `.cue`
> The `--resource` flag in `cue` mode needs a valid Kind, like in above example "Deployment" or StatefulSet, DaemonSet etc.

For a detailed workflow illustrating the capabilities of Cue and Genval for validating and generating Kubernetes configurations, you can refer to [this document](./cmd/cueval/example.md).

### Templates

The `./templates` folder holds some sample files to be used in Genval. the `./templates/inputs` holds JSON input templates for both generating Dockerfiles in container mode and Kubernetes manifests in cue mode. Similarly, all the default policies for both the modes are stored in `./templates/defaultpolices` directory. User can use these template files to start with and as they go along they can customize these files to suite their specific use cases.
The `./templates` folder holds some sample files to be used in Genval. the `./templates/inputs` holds JSON input templates for both generating Dockerfiles in `container` mode and Kubernetes manifests in `cue` mode. Similarly, all the default policies for both the modes are stored in `./templates/defaultpolices` directory. User can use these template files to start with and as they go along they can customize these files to suite their specific use cases.
82 changes: 82 additions & 0 deletions cmd/cueval/example.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
# Validation and generation of Kubernetes Deployment using Genval

## Introduction

In this guide, we'll explore the usage of the Genval tool in Cue mode for validating and generating Kubernetes YAML manifests, specifically focusing on Kubernetes Deployments. Genval simplifies the process of ensuring your Kubernetes configurations adhere to security best practices and industry standards. While we use Kubernetes Deployments as an example, the same principles apply to other Kubernetes resources, ArgoCD, TektonCD, Crossplane, and ClusterAPI resources.

## What is Cue

Cue aka [Cuelang](https://cuelang.org/docs) is a versatile configuration language and runtime environment designed to fulfill various tasks, including defining and validating data schemas, configuring files, and generating configuration artifacts. It presents a succinct and expressive syntax tailored for efficient configuration management while providing advanced features like type checking, constraints, and code generation.


One of Cue's standout features is its robust [type system](https://cuelang.org/docs/tutorials/tour/types/types/), which empowers developers to specify and enforce data types within their configuration files. This type checking capability greatly enhances the reliability and stability of configurations. Additionally, Cue allows developers to establish [constraints](https://cuelang.org/docs/tutorials/tour/intro/constraints/) or rules that configuration data must follow. Enforcing these constraints ensures that configurations align with specific business requirements or predefined rules, elevating the overall quality and correctness of configurations.

## Flow diagram fo Cue mode

<p align="center">
<img src="../cueval.svg" />
</p>




## Genval in action

The core idea behind using Genval in Cue mode is as follows:

- **Cue Schema**: DevOps or platform engineers create a Cue schema that abstracts the complex requirements of a Kubernetes Deployment resource. This schema defines various aspects of the Deployment, such as:

- `securityContext:`
- Resource limits and requests
- any constraints in container images, like no use of `latest` image tags
- Probes like `livenessProbe` and/or `readinessProbe`
- And more

An example Cue schema can be found in `./templates/defaultpolicies/cue/deployment.json`. The Cue format closely resembles JSON. Importing the Kubernetes APIs helps map fields/structs in the `deployment.cue` schema to upstream Kubernetes APIs. For instance:


```cue
package k8s
import (
apps "k8s.io/api/apps/v1"
core "k8s.io/api/core/v1"
)
#Deployment: apps.#Deployment & {
apiVersion: "apps/v1"
kind: "Deployment"
// Removed for brevity
...
}
```

In this example, we map the #Deployment definition to the apps.#Deployment object defined in our cue.mod, which is created by importing the Go APIs into Cue types.

- **Cue Definitions**: Definitions in Cue are marked with a `#` symbol, indicating they are used for validation and not included as data output. The `&` symbol unifies nested fields, mapping them to `#Deployment`.
- **Validation with Input JSON**: Developers provide a minimal input JSON for a Deployment, without worrying about the complex requirements. A sample input JSON for a Deployment is located at `./templates/inputs/cue/deploy.json`.
- **Validation and Generation**: With both the input in JSON format and the Cue schema in place, Genval leverages Cue's powerful validation and generation capabilities. Run the following command:

```shell
$ genval --mode cue --reqinput ./templates/inputs/cue/deploy.json \ # input to be validated
--resource Deployment # a Kind of resource we are trying to validate and generate, needs a valid Kind
--policy ./templates/defaultpolicies/cue/deployment.cue # Our Cue schema/policy for Deployment
--policy ./templates/defaultpolicies/cue/metadata.cue # We can pass multiple policies/schemas referring them in the main policy `deployment.cue`.
```
- `--reqinput`: Specifies the input to be validated.
- `--resource`: Defines the Kubernetes resource kind to validate and generate. It must correspond to a valid Kind.
- `--policy`: Points to the Cue schema/policy for Deployment. Multiple policies/schemas can be used by referencing them in the main policy `deployment.cue`.

- **Validation Result**: If the input passes validation, Genval responds with:

```shell
$ INFO[0000] validation for Deployment succeeded, generated manifest: deployment.yaml
```

The complete, validated deployment.yaml is generated and written to the current working directory.

- **Validation-Only Mode**: If you only want to validate input against a policy without generating a manifest, set the `--verify` flag to true (it's false by default).

By following these steps, you can efficiently validate and generate Kubernetes YAML manifests using Genval in Cue mode, ensuring your configurations meet security requirements and best practices.

53 changes: 0 additions & 53 deletions deployment.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion pkg/utils/utils.go
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ func fetchFileWithCURL(urlStr string) (string, error) {
log.Println("Error checking directory:", err)
}

cmd := exec.Command("curl", "-s", "-o", filepath.Join(dir, filename), urlStr)
cmd := exec.Command("curl", "-s", "-O", filepath.Join(dir, filename), urlStr)
if err := cmd.Run(); err != nil {
log.Errorf("failed fetching content using curl: %v", err)
return "", err
Expand Down
54 changes: 54 additions & 0 deletions templates/defaultpolicies/cue/application.cue
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package argo

import "github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1"

#Application: v1alpha1.#Application & {
apiVersion: string | *"argoproj.io/v1alpha1"
kind: string | *"Application"
metadata: _Metadata
spec: v1alpha.#AppicationSpec & {
source: {
repoURL: string // requires a URL to your manifest repo
targetRevision: string // requires to track the commit/branch/tag
path: string // requires the path to the manifest in the repo
chart: string // requires if your app uses Helm
helm: {
// All your Helm file values go here
...
}
// If your app uses Kustomize overlays, they go here
kustomize: {
...
}
directory: {
...
}
plugin: {
...
}
}

syncPolicy: {
automated: {
...
}
syncOptions: [...string]
retry: {
...
}
...
}
...
revisionHistoryLimit: int
}
}

_Metadata: {
name: *"genval" | string
namespace: *"genval" | string
labels: {
app: string | *"genval"
env: *"mytest" | string
}
...
}
5 changes: 0 additions & 5 deletions templates/defaultpolicies/cue/applicationset.cue

This file was deleted.

39 changes: 37 additions & 2 deletions templates/defaultpolicies/cue/tekton.cue
Original file line number Diff line number Diff line change
@@ -1,5 +1,40 @@
package tekton

import "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1"
import "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1beta1"

#Pipeline: v1.#Pipeline
#Pipeline: v1beta1.#Pipeline & {
apiVersion: string | *"tekton.dev/v1beta1"
kind: string | *"Pipeline"
metadata: _Metadata
spec: v1beta1.#PipelineSpec & {
params: [...{
name: string
description: string
...
}]
tasks: [...{
name: string
taskRef: name: string
params: [...{
name: string
value: string
...
}]
}]
results: [...{
name: string
description: string
value: string
...
}]
}}

_Metadata: {
name: *"genval" | string
namespace: *"genval" | string
labels: {
app: string | *"genval"
env: *"mytest" | string
}
...
}
Loading

0 comments on commit adc3659

Please sign in to comment.