chore: add ai-service charts

ai-service/Chart.yaml

@@ -0,0 +1,6 @@
+apiVersion: v2
+name: ai-service
+description: A Helm chart for Privado AI Service on Kubernetes
+type: application
+version: 0.1.0
+appVersion: "1.16.0"

ai-service/README.md

@@ -0,0 +1,87 @@
+# Charts
+
+The Helm charts for deploying the AI Service. There are two charts for deploying the Ai Service and ChromaDB containers.
+The ChromaDB chart is a dependency for the Ai Service chart.
+
+## Prerequisites
+
+1. In the kubernetes cluster, the following resources should be available:
+    - `g5.xlarge` instance type for the Ai Service container. Or change the instance type in the values.yaml file.
+    - The Ai Service container/pod should be able to connect the Andromeda service.
+    - The Nvidia GPU driver should be installed in the node.
+    - The nvidia-device-plugin should be installed in the cluster.
+        - `kubectl get nodes "-o=custom-columns=NAME:.metadata.name,GPU:.status.allocatable.nvidia\.com/gpu"` to confirm
+          the GPU availability.
+2. In the [values.yaml](./values.yaml) file, verify the following attributes:
+    - Update the following environment variables for the Ai Service container as needed
+        - AI_SERVICE_ANDROMEDA_HOST: "http://andromeda-service.privado.svc.cluster.local:6001"
+        - CHROMA_HOST: "ai-service-chroma-chromadb.privado.svc.cluster.local"
+            - Without any port, the default port (8000) is used.
+            - Without the 'http://' prefix.
+    - NodeSelector configuration
+        - `node.kubernetes.io/instance-type: g5.xlarge` update the instance type as needed.
+            - A GPU core with 24GB+ vRAM is a requirement to deploy the Ai Service container.
+    - Persistent Volume configuration
+        - Update the storage class name as needed.
+        - Update the storage size as needed.
+    - Update the AWS credentials to provide the access key and secret key.
+3. **Updating values.yaml with init.sh.** Instead of manually editing the values.yaml file, the init.sh script can be
+   used to set the necessary values.
+    - To run the script: `bash init.sh`
+    - The script will prompt you for various values. If you do not provide a new value, the default value will be used. 
+      The script will update the values.yaml file with the provided values, ensuring that all necessary configurations 
+      are set correctly. It is also keep a backup of the original values.yaml file. If you want to revert to the 
+      original values.yaml file (or run this script again), you can use the backup file.
+4. **Adding custom annotations and labels.** If you want to add custom annotations and labels to the Ai Service pod, you
+   can do so by updating the values.yaml file. The annotations and labels are added to the pod spec in the deployment
+   template. You can add the annotations and labels under the `annotations` and `labels` sections in the values.yaml file.
+   The annotations and labels should be in key-value format. For example:
+    ```yaml
+    annotations:
+      key1: value1
+      key2: value2
+    labels:
+      key1: value1
+      key2: value2
+    ```
+5. Install ChromaDb helm chart before installing the Ai Service chart using the following commands:
+
+```bash
+   helm repo add chroma https://amikos-tech.github.io/chromadb-chart/
+   helm repo update
+   helm search repo chroma/
+   kubectl create namespace privado
+   helm install ai-service-chroma chroma/chromadb -n privado --set chromadb.allowReset="true" --set chromadb.auth.enabled="false" --set chromadb.serverHost="chromadb"
+```
+
+# Deploying the AI Service
+
+Following steps will deploy ai-service chart:
+
+1. `kubectl config current-context` to verify the kubectl context.
+2. `helm package .` to package the Ai Service chart.
+3. `helm install ai-service ai-service-0.1.0.tgz -n privado` to install the Ai Service chart.
+
+# Post Installation Steps
+
+Following steps will verify the installation of ai-service chart:
+
+1. `kubectl get pods` to verify the Ai Service pods are running.
+2. `kubclt logs -f <pod-name>` to view the container logs of the Ai Service pod.
+3. `kubclt exec -it <pod-name> -- /bin/bash` to access the Ai Service container shell.
+    - `tais` to view the ai-service logs. Following like should be visible:
+        - `GET / => generated X bytes in Y msecs (HTTP/1.1 200) 8 headers in Z bytes`
+
+Following steps will verify successful integration with Privado backend:
+
+1. Login to the Privado dashboard.
+2. Navigate to the "Contract Scanning" menu
+3. Upload a DPA (Data Protection Agreement) pdf. Any public DPA is fine.
+4. Verify that the DPA is scanned and the results are displayed in the dashboard.
+
+# Uninstalling the AI Service
+
+Following steps will uninstall ai-service chart:
+
+1. `helm uninstall ai-service` to uninstall the Ai Service chart.
+2. `helm uninstall ai-service-chroma` to uninstall the ChromaDB chart.

ai-service/init.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+
+# Function to prompt the user for input with a default value
+prompt() {
+  local var_name=$1
+  local prompt_text=$2
+  local default_value=$3
+  read -p "$prompt_text ($default_value): " input
+  echo "${input:-$default_value}"
+}
+
+# File to update
+values_file="values.yaml"
+
+# Prompt the user for each value
+ai_service_andromeda_host=$(prompt "AI_SERVICE_ANDROMEDA_HOST" "Enter the AI Service Andromeda Host URL" "http://andromeda-service.privado.svc.cluster.local:6001")
+chroma_host=$(prompt "CHROMA_HOST" "Enter the Chroma Host (without 'http://' prefix)" "ai-service-chroma-chromadb.privado.svc.cluster.local")
+secret_access_key_id=$(prompt "secretAccessKeyId" "Enter the AWS Secret Access Key ID" "None")
+secret_access_key=$(prompt "secretAccessKey" "Enter the AWS Secret Access Key" "None")
+node_instance_type=$(prompt "node.kubernetes.io/instance-type" "Enter the node instance type" "g5.xlarge")
+models_storage_class=$(prompt "aiServiceModelsPvc.storageClassName" "Enter the storage class name for models PVC" "gp2")
+log_storage_class=$(prompt "aiServiceLogPvc.storageClassName" "Enter the storage class name for log PVC" "gp2")
+
+# Update the values.yaml file
+sed -i.bak -e "s/secretAccessKeyId: .*/secretAccessKeyId: \"$secret_access_key_id\"/" \
+           -e "s/secretAccessKey: .*/secretAccessKey: \"$secret_access_key\"/" \
+           -e "s/node.kubernetes.io\/instance-type: .*/node.kubernetes.io\/instance-type: $node_instance_type/" \
+           -e "s/storageClassName: gp2 # NEEDS-CUSTOMER-INPUT.*/storageClassName: $models_storage_class # NEEDS-CUSTOMER-INPUT/" \
+           -e "s/storageClassName: gp2 # NEEDS-CUSTOMER-INPUT.*/storageClassName: $log_storage_class # NEEDS-CUSTOMER-INPUT/" \
+           -e "s|value: \"http://andromeda-service.privado.svc.cluster.local:6001\" # NEEDS-CUSTOMER-INPUT|value: \"$ai_service_andromeda_host\" # NEEDS-CUSTOMER-INPUT|" \
+           -e "s|value: \"ai-service-chroma-chromadb.privado.svc.cluster.local\"  # NEEDS-CUSTOMER-INPUT \[without 'http://'\ prefix\]|value: \"$chroma_host\"  # NEEDS-CUSTOMER-INPUT \[without 'http://'\ prefix\]|" \
+           "$values_file"
+
+echo "values.yaml has been updated."

ai-service/templates/_helpers.tpl

@@ -0,0 +1,62 @@
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "ai-service.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "ai-service.fullname" -}}
+{{- if .Values.fullnameOverride }}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- $name := default .Chart.Name .Values.nameOverride }}
+{{- if contains $name .Release.Name }}
+{{- .Release.Name | trunc 63 | trimSuffix "-" }}
+{{- else }}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
+{{- end }}
+{{- end }}
+{{- end }}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "ai-service.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
+{{- end }}
+
+{{/*
+Common labels
+*/}}
+{{- define "ai-service.labels" -}}
+helm.sh/chart: {{ include "ai-service.chart" . }}
+{{ include "ai-service.selectorLabels" . }}
+{{- if .Chart.AppVersion }}
+app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
+{{- end }}
+app.kubernetes.io/managed-by: {{ .Release.Service }}
+{{- end }}
+
+{{/*
+Selector labels
+*/}}
+{{- define "ai-service.selectorLabels" -}}
+app.kubernetes.io/name: {{ include "ai-service.name" . }}
+app.kubernetes.io/instance: {{ .Release.Name }}
+{{- end }}
+
+{{/*
+Create the name of the service account to use
+*/}}
+{{- define "ai-service.serviceAccountName" -}}
+{{- if .Values.serviceAccount.create }}
+{{- default (include "ai-service.fullname" .) .Values.serviceAccount.name }}
+{{- else }}
+{{- default "default" .Values.serviceAccount.name }}
+{{- end }}
+{{- end }}

ai-service/templates/cron.update.yaml

@@ -0,0 +1,113 @@
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: {{ .Values.autoupdates.roleName }}
+rules:
+  - apiGroups: [""]
+    resources:
+      - pods
+      - deployments
+    verbs:
+      - 'get'
+      - 'list'
+  - apiGroups: ["extensions", "apps"]
+    resources:
+      - deployments
+    verbs:
+      - 'get'
+      - 'patch'
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: RoleBinding
+metadata:
+  name: {{ .Values.autoupdates.roleBindingName }}
+subjects:
+  - kind: ServiceAccount
+    name: {{ .Values.autoupdates.serviceAccountName }}
+roleRef:
+  kind: Role
+  name: {{ .Values.autoupdates.roleName }}
+  apiGroup: rbac.authorization.k8s.io
+---
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: {{ .Values.autoupdates.serviceAccountName }}
+---
+{{- if .Capabilities.APIVersions.Has "batch/v1/CronJob" }}
+apiVersion: batch/v1
+{{- else }}
+apiVersion: batch/v1beta1
+{{- end }}
+kind: CronJob
+metadata:
+  name: {{ .Values.autoupdates.cronName }}
+  labels:
+    {{- range $key, $val := .Values.autoupdates.cronjobLabels }}
+    {{ $key }}: {{ $val | quote }}
+    {{- end }}
+  annotations:
+    {{- range $key, $val := .Values.autoupdates.cronjobAnnotations }}
+    {{ $key }}: {{ $val | quote }}
+    {{- end }}
+spec:
+  schedule: {{ .Values.autoupdates.cronSchedule | quote }}
+  successfulJobsHistoryLimit: 1
+  failedJobsHistoryLimit: 1
+  jobTemplate:
+    spec:
+      template:
+        spec:
+          serviceAccountName: {{ .Values.autoupdates.serviceAccountName }}
+          restartPolicy: {{ .Values.autoupdates.cronPodRestartPolicy }}
+          containers:
+            - name: updater
+              image: {{ .Values.autoupdates.image.name }}:{{ .Values.autoupdates.image.tag }}
+              imagePullPolicy: {{ .Values.image.pullPolicy }}
+              command:
+                - /bin/bash
+                - -c
+                - |-
+                  set -e
+                  NAMESPACE={{ .Release.Namespace }}
+                  echo "Checking for image updates on AWS ECR for {{ .Values.autoupdates.deploymentName }}"
+                  # Function to get AWS ECR image SHA
+                  function get_ecr_sha() {
+                    REPO_NAME=$1
+                    TAG=$2
+                    IMAGE_SHA=`aws ecr describe-images --repository-name ${REPO_NAME} --region ${AWS_REGION} --image-ids imageTag=${TAG} | jq -r ".imageDetails[0].imageDigest"`
+
+                    echo $IMAGE_SHA
+                  }
+
+                  # Function to check and patch deployment if there's an image SHA mismatch
+                  function check_and_patch_deployment() {
+                    DEPLOYMENT_NAME=$1
+                    POD_APP_LABEL=$2
+                    IMAGE_NAME=$3
+                    IMAGE_TAG=$4
+
+                    POD=`kubectl -n $NAMESPACE get pods -l "app.kubernetes.io/name=$POD_APP_LABEL" -o name | head -1`
+                    CURRENT_IMAGE_SHA=`kubectl -n $NAMESPACE get $POD -o jsonpath="{..imageID}" | cut -f 2 -d "@"`
+                    LATEST_IMAGE_SHA=$(get_ecr_sha $IMAGE_NAME $IMAGE_TAG)
+
+                    echo "> Current $DEPLOYMENT_NAME SHA: $CURRENT_IMAGE_SHA"
+                    echo "> Latest $DEPLOYMENT_NAME SHA: $LATEST_IMAGE_SHA"
+                    if [[ $CURRENT_IMAGE_SHA != $LATEST_IMAGE_SHA ]]; then
+                      echo "> Update for $POD found"
+                      echo "> Applying patch to $DEPLOYMENT_NAME"
+                      kubectl -n $NAMESPACE patch deployment $DEPLOYMENT_NAME -p "{\"spec\": {\"template\":{\"metadata\":{\"annotations\":{\"{{ .Values.autoupdates.annotationName }}\":\"'$LATEST_IMAGE_SHA'\"}}}} }"
+                    fi
+                  }
+
+                  check_and_patch_deployment "{{ .Values.autoupdates.deploymentName }}" "{{ .Values.autoupdates.podAppLabel }}" "{{ splitList "/" (tpl .Values.image.repository .) | last }}" "{{ .Values.image.tag }}"
+              envFrom:
+              - secretRef:
+                  name: {{ .Values.image.imagePullSecret }}
+              env:
+                - name: SECRET_NAME
+                  value: {{ .Values.image.imagePullSecret }}
+                - name: AWS_REGION
+                  value: {{ .Values.autoupdates.awsRegion }}
+                - name: NAMESPACE
+                  value: {{ .Release.Namespace }}

ai-service/templates/deployment.yaml

@@ -0,0 +1,68 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: {{ include "ai-service.fullname" . }}
+  labels:
+    {{- include "ai-service.labels" . | nindent 4 }}
+spec:
+  {{- if not .Values.autoscaling.enabled }}
+  replicas: {{ .Values.replicaCount }}
+  {{- end }}
+  selector:
+    matchLabels:
+      {{- include "ai-service.selectorLabels" . | nindent 6 }}
+  template:
+    metadata:
+      {{- with .Values.podAnnotations }}
+      annotations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      labels:
+        {{- include "ai-service.labels" . | nindent 8 }}
+        {{- with .Values.podLabels }}
+        {{- toYaml . | nindent 8 }}
+        {{- end }}
+    spec:
+      imagePullSecrets:
+        - name: {{ .Values.image.imagePullSecret }}
+      serviceAccountName: {{ include "ai-service.serviceAccountName" . }}
+      securityContext:
+        {{- toYaml .Values.podSecurityContext | nindent 8 }}
+      containers:
+        - name: {{ .Chart.Name }}
+          securityContext:
+            {{- toYaml .Values.securityContext | nindent 12 }}
+          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
+          imagePullPolicy: {{ .Values.image.pullPolicy }}
+          ports:
+            - name: http
+              containerPort: {{ .Values.service.port }}
+              protocol: TCP
+          env:
+            {{- toYaml .Values.service.env | nindent 12 }}
+          livenessProbe:
+            {{- toYaml .Values.livenessProbe | nindent 12 }}
+          readinessProbe:
+            {{- toYaml .Values.readinessProbe | nindent 12 }}
+          resources:
+            {{- toYaml .Values.resources | nindent 12 }}
+          {{- with .Values.volumeMounts }}
+          volumeMounts:
+            {{- toYaml . | nindent 12 }}
+          {{- end }}
+      {{- with .Values.volumes }}
+      volumes:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.nodeSelector }}
+      nodeSelector:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.affinity }}
+      affinity:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}
+      {{- with .Values.tolerations }}
+      tolerations:
+        {{- toYaml . | nindent 8 }}
+      {{- end }}