Wal 10593 generic async driver (#118)

resource-definitions/generic-async-driver/README.md

@@ -0,0 +1,8 @@
+## Resource Definitions using the Generic Async Driver
+
+This section contains example Resource Definitions using the [Generic Async Driver](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/).
+
+The requirements to make these Resource Definitions work with the Orchestrator are:
+
+* The image supplied in the Generic Async Driver Definitions in `values.job.image` should adhere to the [interface between Driver and Runner Image](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#contract-between-generic-async-driver-and-runner-image).
+* The cluster chosen to run the Kubernetes Job should be [properly configured](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#cluster-object).

resource-definitions/generic-async-driver/inline-terraform/README.md

@@ -0,0 +1,13 @@
+The [Generic Async Driver](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/) executes a container supplied as input as part of a Kubernetes Job execution in a target Kubernetes cluster.
+
+The example in this section shows:
+
+* How to reference a `config` Resource Definition to provide the data needed to create a Kubernetes Job in the desired [cluster](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#cluster-object).
+* How to reference a `config` Resource Definition to create the [job](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#job-object) with the proper configuration.
+* How to make the Kubernetes Job able to [pull an image from a private registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry).
+* How to inject the cloud account credentials into the IaC code running in the container via the [credentials_config](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#credentials_config-object) object.
+
+The example is made up out of these files:
+* [k8s-cluster-runner-config.yaml](k8s-cluster-runner-config.yaml): provides a connection to a [GKE cluster](https://developer.humanitec.com/integration-and-extensions/drivers/k8-drivers/gke-cluster/).
+* [agent-runner.yaml](agent-runner.yaml): provides the configuration to access a private cluster via the Humanitec Agent.
+* [s3.yaml](s3.yaml): in addition to referencing the `config` Resource Definition, it defines the Terraform scripts to run to provision an S3 bucket whose name is produced appending a random postfix to the application and the environment name. The supplied scripts provide an AWS S3 bucket as place where to store the resource state.

resource-definitions/generic-async-driver/inline-terraform/agent-runner.yaml

@@ -0,0 +1,16 @@
+# This Resource Definition specifies the Humanitec Agent to use for the Runner.
+apiVersion: entity.humanitec.io/v1b1
+kind: Definition
+metadata:
+  id: agent-runner
+entity:
+  driver_type: humanitec/agent
+  name: agent-runner
+  type: agent
+  driver_inputs:
+    values:
+      id: my-agent
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+      class: runner      

resource-definitions/generic-async-driver/inline-terraform/k8s-cluster-runner-config.yaml

@@ -0,0 +1,45 @@
+# This Resource Definition provides configuration values for the Generic Async Driver.
+apiVersion: entity.humanitec.io/v1b1
+kind: Definition
+metadata:
+  id: qa-testing-ground-generic-async
+entity:
+  name: qa-testing-ground-generic-async
+  type: config
+  driver_type: humanitec/echo
+  driver_inputs:
+    values:
+      job:
+        # Change to match the image you built to run the IaC of your choice
+        image: ghcr.io/my-registry/generic-async-driver-runner:1.0.1
+        # Change to match the command to run your image or remove it if you want to use the image entrypoint
+        command: ["/opt/container"]
+        # Change to match the mount point of your shared directory
+        shared_directory: /home/runneruser/workspace
+        # Change to the namespace name you created to host the Kubernetes Job created by the Driver.
+        namespace: humanitec-runner
+        # Change to the service account name with permissions to create secrets/configmaps in the Kubernetes Job namespace you created.
+        service_account: humanitec-runner-job
+        # This assumes a secret with the given name exists in the desired namespace and it contains the credentials to pull the job image from the private registry.
+        pod_template: |
+          spec:
+            imagePullSecrets:
+              - name: ghcr-private-registry
+      # Change to match the configuration of your target cluster
+      cluster:
+        cluster_type: gke
+        account: my-org/my-gcp-cloud-account
+        cluster:
+          loadbalancer: 10.10.10.10
+          name: my-cluster
+          project_id: my-project
+          zone: europe-west2
+          internal_ip: true
+    # Change to match the desired agent (if any)
+    secret_refs:
+      agent_url: 
+        value: ${resources['agent.default#agent'].outputs.url}
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+      class: runner

resource-definitions/generic-async-driver/inline-terraform/s3.yaml

@@ -0,0 +1,92 @@
+# This Resource Definition specifies an `s3` Resource to be provisioned through inline Terraform code.
+apiVersion: entity.humanitec.io/v1b1
+
+kind: Definition
+metadata:
+  id: aws-s3
+entity:
+  name: aws-s3
+  type: s3
+  driver_type: humanitec/generic-async
+  driver_account: my-aws-cloud-account
+  driver_inputs:
+    values:
+      job: ${resources['config.runner'].outputs.job}
+      cluster:
+        cluster_type: ${resources['config.runner'].outputs.cluster.cluster_type}
+        account: ${resources['config.runner'].outputs.cluster.account}
+        cluster: ${resources['config.runner'].outputs.cluster.cluster}
+      # Needed to authenticate to aws TF provider in the TF code passed via files inputs
+      credentials_config:
+        environment:
+          AWS_ACCESS_KEY_ID: AccessKeyId
+          AWS_SECRET_ACCESS_KEY: SecretAccessKey
+      files:
+        terraform.tfvars.json:  | 
+          {"REGION": "eu-west-3", "BUCKET": "${context.app.id}-${context.env.id}"}
+        # Change to match the backend of your choice.
+        backend.tf: |
+          terraform {
+            backend "s3" {
+              bucket = "my-s3-to-store-tf-state"
+              key = "${context.res.guresid}/state/terraform.tfstate"
+              region = "eu-west-3"
+            }
+          }
+        providers.tf: |
+          terraform {
+            required_providers {
+              aws = {
+                source = "hashicorp/aws"
+                version = "~> 5.72.0"
+              }
+            }
+          }
+        vars.tf: |
+          variable "REGION" {
+              type = string
+          }
+
+          variable "BUCKET" {
+              type = string
+          }
+        main.tf: |
+          provider "aws" {
+            region     = var.REGION
+            default_tags {
+              tags = {
+                CreatedBy = "Humanitec"
+              }
+            }
+          }
+
+          resource "random_string" "bucket_suffix" {
+            length           = 5
+            special          = false
+            upper            = false
+          }
+
+          module "aws_s3" {
+            source = "terraform-aws-modules/s3-bucket/aws"
+            bucket = format("%s-%s", var.BUCKET, random_string.bucket_suffix.result)
+            acl    = "private"
+            force_destroy = true
+            control_object_ownership = true
+            object_ownership         = "BucketOwnerPreferred"
+          }
+
+          output "region" {
+            value = module.aws_s3.s3_bucket_region
+          }
+
+          output "bucket" {
+            value = module.aws_s3.s3_bucket_id
+          }
+    secret_refs:
+      cluster:
+        agent_url: 
+          value: ${resources['config.runner'].outputs.agent_url}
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+

resource-definitions/generic-async-driver/private-git-repo/README.md

@@ -0,0 +1,14 @@
+The [Generic Async Driver](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/) executes a container supplied as input as part of a Kubernetes Job execution in a target Kubernetes cluster.
+
+The example in this section shows:
+
+* How to reference a `config` Resource Definition to provide the data needed to create a Kubernetes Job in the desired [cluster](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#cluster-object).
+* How to reference a `config` Resource Definition to create the [job](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#job-object) with the proper configuration.
+* How to make the Kubernetes Job able to [pull an image from a private registry](https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry).
+* How to inject the cloud account credentials into the IaC code running in the container via the [credentials_config](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#credentials_config-object) object.
+* How to fetch the IaC scripts from a private Repository, via [non-secret](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#source-object) and [secret](https://developer.humanitec.com/integration-and-extensions/drivers/generic-drivers/generic-async/#source-object-1) fields.
+
+The example is made up out of these files:
+* [k8s-cluster-runner-config.yaml](k8s-cluster-runner-config.yaml): provides a connection to a [GKE cluster](https://developer.humanitec.com/integration-and-extensions/drivers/k8-drivers/gke-cluster/).
+* [agent-runner.yaml](agent-runner.yaml): provides the configuration to access a private cluster via the Humanitec Agent.
+* [s3.yaml](s3.yaml): in addition to referencing the `config` Resource Definition, it defines how to fetch the Terraform scripts from a private Github Repository to provision an S3 bucket. This also provides via file an AWS S3 bucket as place where to store the resource state.

resource-definitions/generic-async-driver/private-git-repo/agent-runner.yaml

@@ -0,0 +1,16 @@
+# This Resource Definition specifies the Humanitec Agent to use for the Runner.
+apiVersion: entity.humanitec.io/v1b1
+kind: Definition
+metadata:
+  id: agent-runner
+entity:
+  driver_type: humanitec/agent
+  name: agent-runner
+  type: agent
+  driver_inputs:
+    values:
+      id: my-agent
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+      class: runner      

resource-definitions/generic-async-driver/private-git-repo/k8s-cluster-runner-config.yaml

@@ -0,0 +1,45 @@
+# This Resource Definition provides configuration values for the Generic Async Driver.
+apiVersion: entity.humanitec.io/v1b1
+kind: Definition
+metadata:
+  id: qa-testing-ground-generic-async
+entity:
+  name: qa-testing-ground-generic-async
+  type: config
+  driver_type: humanitec/echo
+  driver_inputs:
+    values:
+      job:
+        # Change to match the image you built to run the IaC of your choice
+        image: ghcr.io/my-registry/generic-async-driver-runner:1.0.1
+        # Change to match the command to run your image or remove it if you want to use the image entrypoint
+        command: ["/opt/container"]
+        # Change to match the mount point of your shared directory
+        shared_directory: /home/runneruser/workspace
+        # Change to the namespace name you created to host the Kubernetes Job created by the Driver.
+        namespace: humanitec-runner
+        # Change to the service account name with permissions to create secrets/configmaps in the Kubernetes Job namespace you created.
+        service_account: humanitec-runner-job
+        # This assumes a secret with the given name exists in the desired namespace and it contains the credentials to pull the job image from the private registry.
+        pod_template: |
+          spec:
+            imagePullSecrets:
+              - name: ghcr-private-registry
+      # Change to match the configuration of your target cluster
+      cluster:
+        cluster_type: gke
+        account: my-org/my-gcp-cloud-account
+        cluster:
+          loadbalancer: 10.10.10.10
+          name: my-cluster
+          project_id: my-project
+          zone: europe-west2
+          internal_ip: true
+    # Change to match the desired agent (if any)
+    secret_refs:
+      agent_url: 
+        value: ${resources['agent.default#agent'].outputs.url}
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+      class: runner

resource-definitions/generic-async-driver/private-git-repo/s3.yaml

@@ -0,0 +1,53 @@
+# This Resource Definition specifies an `s3` Resource to be provisioned through Terraform code read from a private Git repository accessed via an SSH key.
+apiVersion: entity.humanitec.io/v1b1
+
+kind: Definition
+metadata:
+  id: aws-s3
+entity:
+  name: aws-s3
+  type: s3
+  driver_type: humanitec/generic-async
+  driver_account: my-aws-cloud-account
+  driver_inputs:
+    values:
+      job: ${resources['config.runner'].outputs.job}
+      cluster:
+        cluster_type: ${resources['config.runner'].outputs.cluster.cluster_type}
+        account: ${resources['config.runner'].outputs.cluster.account}
+        cluster: ${resources['config.runner'].outputs.cluster.cluster}
+      # Needed to authenticate to aws TF provider in the TF code passed via files inputs
+      credentials_config:
+        environment:
+          AWS_ACCESS_KEY_ID: AccessKeyId
+          AWS_SECRET_ACCESS_KEY: SecretAccessKey
+      # Change to match your repository
+      source:
+        path: path/to/my/iac/scripts
+        ref: refs/heads/main
+        url: [email protected]:my-org/my-repo.git          
+      files:
+        terraform.tfvars.json:  | 
+          {"REGION": "eu-west-3", "BUCKET": "${context.app.id}-${context.env.id}"}
+        # Change to match the backend of your choice.
+        backend.tf: |
+          terraform {
+            backend "s3" {
+              bucket = "my-s3-to-store-tf-state"
+              key = "${context.res.guresid}/state/terraform.tfstate"
+              region = "eu-west-3"
+            }
+          }
+    secret_refs:
+      cluster:
+        agent_url: 
+          value: ${resources['config.runner'].outputs.agent_url}
+      # Change to match where your ssh key is stored
+      source:
+        ssh_key:
+          store: my-secret-store
+          ref: my-path-to-git-ssh-key
+  criteria:
+    # Change to match the name of the development type you want this to apply to
+    - env_type: development
+