diff --git a/.github/workflows/repository.yml b/.github/workflows/repository.yml index 1a6da25d..de8842fa 100644 --- a/.github/workflows/repository.yml +++ b/.github/workflows/repository.yml @@ -3,9 +3,6 @@ on: push: branches: - main - pull_request: - branches: - - main release: types: [published, created] diff --git a/README.md b/README.md index 6322dcd2..f4b29e1f 100644 --- a/README.md +++ b/README.md @@ -1,20 +1,20 @@ # Charts -[Helm](https://github.com/helm/helm) Charts for Open Data Platform +[Helm](https://github.com/helm/helm) Charts for Goto company ecosystem. -Charts are curated application definitions for [Helm](https://github.com/helm/helm). Use this repository to submit official Open Data Platform helm Charts. For more information about installing and using Helm, see it's `README.md`. +Charts are curated application definitions for [Helm](https://github.com/helm/helm). Use this repository to submit official Goto company helm Charts. For more information about installing and using Helm, see it's `README.md`. ## TL;DR ```bash -$ helm repo add odpf https://odpf.github.io/charts/ -$ helm search repo odpf -$ helm install my-release odpf/ +$ helm repo add gotocompany https://goto.github.io/charts/ +$ helm search repo gotocompany +$ helm install my-release gotocompany/ ``` ## Repository Structure -This repository contains the source for the packaged and versioned charts (released in the [Github Pages](https://odpf.github.io/charts/)). -The Charts in the `stable/` directory in the main branch match the latest packaged Chart in the [Chart Repository](https://odpf.github.io/charts/). +This repository contains the source for the packaged and versioned charts (released in the [Github Pages](https://goto.github.io/charts/)). +The Charts in the `stable/` directory in the main branch match the latest packaged Chart in the [Chart Repository](https://goto.github.io/charts/). There may be previous versions of a Chart available in the Chart Repository. The purpose of this repository is to provide a place for maintaining and contributing official Charts, with CI processes in place for managing the releasing of Charts into the Chart Repository. @@ -23,21 +23,21 @@ Stable Charts meet the criteria in the technical requirements. ## Usage -Once you have installed the Helm client on your kubernetes cluster, you can deploy a ODPF Helm Chart into a Kubernetes cluster. +Once you have installed the Helm client on your kubernetes cluster, you can deploy a Helm Chart into a Kubernetes cluster. Please refer to the [Quick Start guide](https://helm.sh/docs/intro/quickstart/) if you wish to get running in just a few commands, otherwise the [Using Helm Guide](https://helm.sh/docs/intro/using_helm/) provides detailed instructions on how to use the Helm client to manage packages on your Kubernetes cluster. -Helm Repo URL: `https://odpf.github.io/charts/` +Helm Repo URL: `https://goto.github.io/charts/` ```bash -# To add ODPF Helm repo -$ helm repo add odpf https://odpf.github.io/charts/ +# To add gotocompany Helm repo +$ helm repo add gotocompany https://goto.github.io/charts/ # To list all the charts in the repo -$ helm search repo odpf +$ helm search repo gotocompany # To install any searched chart -$ helm install my-release odpf/ +$ helm install my-release gotocompany/ ``` ## Contributing a Chart diff --git a/stable/app/Chart.yaml b/stable/app/Chart.yaml index 0af83a9b..0311583f 100644 --- a/stable/app/Chart.yaml +++ b/stable/app/Chart.yaml @@ -15,4 +15,4 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.4.3 +version: 0.7.0 diff --git a/stable/app/README.md b/stable/app/README.md index 5e803277..e965197a 100644 --- a/stable/app/README.md +++ b/stable/app/README.md @@ -17,11 +17,11 @@ This chart can be used for deploying any app on kubenetes using helm To install the chart with the release name `my-release` and image `siren`: ```bash -$ helm install --name my-release . -f ./path-to/values.yaml --set image.repository=odpf/siren --set image.tag=latest --set ingress.enabled=true +$ helm install --name my-release . -f ./path-to/values.yaml --set image.repository=gotocompany/siren --set image.tag=latest --set ingress.enabled=true ``` -The command deploys odpf/siren on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. +The command deploys gotocompany/siren on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. > **Tip**: List all releases using `helm list` @@ -55,10 +55,12 @@ The following table lists the configurable parameters of the Siren chart and the | container.readinessProbe.httpGet.port | string | `"tcp"` | | | cron.enabled | bool | `false` | | | cron.jobs[0].name | string | `""` | | +| cron.jobs[0].serviceAccountName | string | `""` | | | cron.jobs[0].command | string | `""` | | | cron.jobs[0].args | string | `""` | | | cron.jobs[0].schedule | string | `"0 0 * * *"` | | -| cron.jobs[0].restartPolicy | string | `Never` | | +| cron.jobs[0].restartPolicy | string | `Never` | +| cron.jobs[0].resources | object | {} | fullnameOverride | string | `""` | | | image.pullPolicy | string | `"IfNotPresent"` | Specify the docker image path/repository. | | image.repository | string | `nil` | | diff --git a/stable/app/templates/cron.yaml b/stable/app/templates/cron.yaml index 10a50ece..889d5353 100644 --- a/stable/app/templates/cron.yaml +++ b/stable/app/templates/cron.yaml @@ -2,11 +2,12 @@ {{- $fullName := include "app.fullname" . -}} {{- $appLabels := include "app.labels" . -}} {{- $appSelectorLabels := include "app.selectorLabels" . -}} +{{- $podLabels := .Values.podLabels }} {{- range $i, $job := .Values.cron.jobs }} apiVersion: batch/v1 kind: CronJob metadata: - name: "{{ $fullName | trunc 24 }}-{{ $i }}-{{ $job.name | trunc 24}}" + name: "{{ ($fullName | trunc 24 | trimSuffix "-") }}-{{ $i }}-{{ ($job.name | trunc 24 | trimSuffix "-") }}" namespace: {{ $.Release.Namespace }} labels: {{- $appLabels | nindent 4 }} @@ -18,22 +19,116 @@ spec: template: metadata: labels: - {{- $appSelectorLabels | nindent 12 }} + {{- $appSelectorLabels | nindent 12 }} + {{- with $podLabels }} + {{- toYaml . | nindent 12 }} + {{- end }} spec: + {{- if $job.serviceAccountName }} + serviceAccountName: "{{ $job.serviceAccountName }}" + {{- end }} + {{- if $job.initContainers }} + initContainers: + {{- range $initContainer := $job.initContainers }} + - {{- toYaml $initContainer | nindent 14 }} + {{- end }} + {{- end }} containers: - name: {{ $job.name }} image: "{{ $.Values.image.repository }}:{{ $.Values.image.tag | default $.Chart.AppVersion }}" imagePullPolicy: {{ $.Values.image.pullPolicy }} command: - {{- toYaml $job.command | nindent 16 }} + {{- if $.Values.otelcollector.enabled }} + - /bin/sh + - -c + - | + {{ join " " $job.command }} + pkill -SIGTERM otelcol-contrib + {{- else }} + {{- toYaml $job.command | nindent 16 }} + {{- end }} args: {{- toYaml $job.args | nindent 16 }} envFrom: - configMapRef: name: {{ $fullName }} + {{- if $.Values.secretConfig }} - secretRef: name: {{ $fullName }}-secret + {{- end }} + {{- if $job.volumeMounts }} + volumeMounts: + {{- toYaml $job.volumeMounts | nindent 16 }} + {{- end }} + resources: + {{- toYaml $job.resources | nindent 16 }} + {{- if $.Values.otelcollector.enabled }} + - name: otel-collector-sidecar + image: {{ $.Values.otelcollector.image }} + imagePullPolicy: IfNotPresent + args: + - "--config=/etc/otelcollector/config.yaml" + ports: + - containerPort: 1888 + name: pprof + protocol: TCP + - containerPort: 8888 + name: prometheusself + protocol: TCP + - containerPort: 8889 + name: prometheus + protocol: TCP + - containerPort: 13133 + name: healthcheck + protocol: TCP + - containerPort: 4317 + name: otlpgrpcrecv + protocol: TCP + - containerPort: 4318 + name: otlphttprecv + protocol: TCP + - containerPort: 55679 + name: zpages + protocol: TCP + env: + - name: K8S_NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: K8S_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: K8S_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: K8S_POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: OTEL_RESOURCE_ATTRIBUTES + value: | + k8s.deployment.name={{ $job.name }},k8s.namespace.name=$(K8S_POD_NAMESPACE),k8s.node.name=$(K8S_NODE_NAME),k8s.pod.name=$(K8S_POD_NAME),k8s.pod.ip=$(K8S_POD_IP) + volumeMounts: + - name: otelcollector-conf + mountPath: /etc/otelcollector/ + {{- if $.Values.otelcollector.resources }} + resources: + {{- toYaml $.Values.otelcollector.resources | nindent 16 }} + {{- end }} + shareProcessNamespace: true + {{- end}} restartPolicy: {{ $job.restartPolicy }} + volumes: + {{- range $volume := $job.volumes }} + - {{- toYaml $volume | nindent 14 }} + {{- end }} + {{- if $.Values.otelcollector.enabled }} + - name: otelcollector-conf + configMap: + name: {{ $fullName }}-otelcollector + {{- end }} --- {{- end -}} {{- end -}} \ No newline at end of file diff --git a/stable/app/templates/deployment.yaml b/stable/app/templates/deployment.yaml index cdfa5abf..da65a7af 100644 --- a/stable/app/templates/deployment.yaml +++ b/stable/app/templates/deployment.yaml @@ -1,3 +1,4 @@ +{{- if .Values.container.enabled }} apiVersion: apps/v1 kind: Deployment metadata: @@ -21,13 +22,25 @@ spec: {{- end }} labels: {{- include "app.selectorLabels" . | nindent 8 }} + {{- with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} spec: + {{- if .Values.serviceAccountName }} + serviceAccountName: {{ .Values.serviceAccountName }} + {{- end }} {{- with .Values.imagePullSecrets }} imagePullSecrets: {{- toYaml . | nindent 8 }} {{- end }} securityContext: {{- toYaml .Values.podSecurityContext | nindent 8 }} + {{- if gt (len .Values.initContainers) 0 }} + initContainers: + {{- range $initContainer := .Values.initContainers }} + - {{- toYaml $initContainer | nindent 10 }} + {{- end }} + {{- end }} containers: - name: {{ .Chart.Name }} securityContext: @@ -65,10 +78,87 @@ spec: envFrom: - configMapRef: name: "{{ include "app.fullname" . }}" - {{- if .Values.secretConfig }} + {{- if .Values.secretConfig }} - secretRef: name: "{{ template "app.fullname" . }}-secret" {{- end }} + {{- if .Values.otelcollector.enabled }} + - name: otel-collector-sidecar + image: {{ .Values.otelcollector.image }} + imagePullPolicy: IfNotPresent + args: + - "--config=/etc/otelcollector/config.yaml" + ports: + - containerPort: 1888 + name: pprof + protocol: TCP + - containerPort: 8888 + name: prometheusself + protocol: TCP + - containerPort: 8889 + name: prometheus + protocol: TCP + - containerPort: 13133 + name: healthcheck + protocol: TCP + - containerPort: 4317 + name: otlpgrpcrecv + protocol: TCP + - containerPort: 4318 + name: otlphttprecv + protocol: TCP + - containerPort: 55679 + name: zpages + protocol: TCP + {{- if .Values.otelcollector.useEnvFromConfig }} + envFrom: + - configMapRef: + name: "{{ include "app.fullname" . }}" + {{- if .Values.secretConfig }} + - secretRef: + name: "{{ template "app.fullname" . }}-secret" + {{- end }} + {{- end }} + env: + - name: K8S_NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: K8S_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: K8S_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: K8S_POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: OTEL_RESOURCE_ATTRIBUTES + value: | + k8s.deployment.name={{ include "app.fullname" . }},k8s.namespace.name=$(K8S_POD_NAMESPACE),k8s.node.name=$(K8S_NODE_NAME),k8s.pod.name=$(K8S_POD_NAME),k8s.pod.ip=$(K8S_POD_IP) + volumeMounts: + - name: otelcollector-conf + mountPath: /etc/otelcollector/ + {{- if .Values.otelcollector.resources }} + resources: + {{- toYaml .Values.otelcollector.resources | nindent 12 }} + {{- end }} + readinessProbe: + httpGet: + path: /readyz + port: 13133 + initialDelaySeconds: {{ .Values.otelcollector.readinessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.otelcollector.readinessProbe.periodSeconds }} + livenessProbe: + httpGet: + path: /healthz + port: 13133 + initialDelaySeconds: {{ .Values.otelcollector.livenessProbe.initialDelaySeconds }} + periodSeconds: {{ .Values.otelcollector.livenessProbe.periodSeconds }} + {{- end }} {{- if .Values.telegraf.enabled }} - name: telegraf-sidecar image: {{ .Values.telegraf.image }} @@ -81,6 +171,10 @@ spec: - {{ . }} {{- end }} imagePullPolicy: IfNotPresent + {{- if .Values.telegraf.resources }} + resources: + {{- toYaml .Values.telegraf.resources | nindent 12 }} + {{- end }} ports: - containerPort: {{ .Values.telegraf.containerPort }} name: http @@ -89,15 +183,59 @@ spec: - name: telegraf-conf mountPath: /etc/telegraf/ {{- end }} + {{- if .Values.iap.enabled }} + - name: iap-sidecar + image: {{ .Values.iap.image }} + imagePullPolicy: IfNotPresent + ports: + - containerPort: {{ .Values.iap.containerPort }} + name: http + protocol: TCP + readinessProbe: + tcpSocket: + port: {{ .Values.iap.containerPort }} + initialDelaySeconds: {{ .Values.iap.livenessProbe.initialDelaySeconds }} + timeoutSeconds: {{ .Values.iap.livenessProbe.timeoutSeconds }} + livenessProbe: + tcpSocket: + port: {{ .Values.iap.containerPort }} + initialDelaySeconds: {{ .Values.iap.livenessProbe.initialDelaySeconds }} + timeoutSeconds: {{ .Values.iap.livenessProbe.timeoutSeconds }} + {{- if .Values.iap.resources }} + resources: + {{- toYaml .Values.iap.resources | nindent 12 }} + {{- end }} + env: + - name: REFRESH_TIME_SECONDS + value: {{ .Values.iap.refreshTimeSeconds }} + - name: SERVICE_ACCOUNT_CREDENTIALS + value: {{ .Values.iap.serviceAccountMountPath }}/{{ .Values.iap.serviceAccountFileName }} + - name: IAP_HOST + value: {{ .Values.iap.host }} + - name: CLIENT_ID + value: {{ .Values.iap.clientID }} + - name: LOGGER_LEVEL + value: {{ .Values.iap.logLevel }} + - name: PORT + value: "{{ .Values.iap.containerPort }}" + volumeMounts: + - mountPath: {{ .Values.iap.serviceAccountMountPath }} + name: {{ .Values.iap.serviceAccountSecret }} + {{- end }} volumes: {{- range $volume := .Values.volumes }} - - {{- toYaml $volume | nindent 12 }} + - {{- toYaml $volume | nindent 10 }} {{- end }} {{- if .Values.telegraf.enabled }} - name: telegraf-conf configMap: name: {{ template "app.fullname" . }}-telegraf {{- end }} + {{- if .Values.otelcollector.enabled }} + - name: otelcollector-conf + configMap: + name: {{ template "app.fullname" . }}-otelcollector + {{- end }} {{- with .Values.nodeSelector }} nodeSelector: {{- toYaml . | nindent 8 }} @@ -109,4 +247,5 @@ spec: {{- with .Values.tolerations }} tolerations: {{- toYaml . | nindent 8 }} - {{- end }} \ No newline at end of file + {{- end }} +{{- end -}} diff --git a/stable/app/templates/hpa.yaml b/stable/app/templates/hpa.yaml new file mode 100644 index 00000000..dd167158 --- /dev/null +++ b/stable/app/templates/hpa.yaml @@ -0,0 +1,41 @@ +{{- if .Values.hpa.enabled }} +{{- $fullName := include "app.fullname" . -}} +{{- $appLabels := include "app.labels" . -}} +apiVersion: autoscaling/v2 +kind: HorizontalPodAutoscaler +metadata: + name: {{ $fullName }} + namespace: {{ $.Release.Namespace }} + labels: + {{- $appLabels | nindent 4 }} +spec: + minReplicas: {{ .Values.hpa.replicas.min }} + maxReplicas: {{ .Values.hpa.replicas.max }} + scaleTargetRef: + apiVersion: apps/v1 + kind: Deployment + name: {{ $fullName }} + behavior: + scaleDown: + stabilizationWindowSeconds: {{ .Values.hpa.stabilizationWindowSeconds.scaleDown }} + scaleUp: + stabilizationWindowSeconds: {{ .Values.hpa.stabilizationWindowSeconds.scaleUp }} + metrics: + {{- if .Values.hpa.targetUtilization.memory }} + - type: Resource + resource: + name: memory + target: + type: Utilization + averageUtilization: {{ .Values.hpa.targetUtilization.memory }} + {{- end }} + {{- if .Values.hpa.targetUtilization.cpu }} + - type: Resource + resource: + name: cpu + target: + type: Utilization + averageUtilization: {{ .Values.hpa.targetUtilization.cpu }} + {{- end }} +--- +{{- end -}} \ No newline at end of file diff --git a/stable/app/templates/ingress.yaml b/stable/app/templates/ingress.yaml index b083ec00..67ed1af3 100644 --- a/stable/app/templates/ingress.yaml +++ b/stable/app/templates/ingress.yaml @@ -1,28 +1,18 @@ {{- if .Values.ingress.enabled -}} {{- $fullName := include "app.fullname" . -}} -{{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }} - {{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }} - {{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}} - {{- end }} -{{- end }} -{{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}} +{{- $labels := include "app.labels" . -}} apiVersion: networking.k8s.io/v1 -{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}} -apiVersion: networking.k8s.io/v1beta1 -{{- else -}} -apiVersion: extensions/v1beta1 -{{- end }} kind: Ingress metadata: name: {{ $fullName }} labels: - {{- include "app.labels" . | nindent 4 }} + {{- $labels | nindent 4 }} {{- with .Values.ingress.annotations }} annotations: {{- toYaml . | nindent 4 }} {{- end }} spec: - {{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }} + {{- if .Values.ingress.className }} ingressClassName: {{ .Values.ingress.className }} {{- end }} {{- if .Values.ingress.tls }} @@ -43,20 +33,56 @@ spec: paths: {{- range .paths }} - path: {{ .path }} - {{- if and .pathType (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }} pathType: {{ .pathType }} - {{- end }} backend: - {{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }} service: name: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} port: number: {{ .backend.service.port.number }} - {{- else }} - serviceName: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} - servicePort: {{ .backend.service.port.number }} - {{- end }} {{- end }} {{- end }} - +{{- if .Values.ingress.additionalIngress }} + {{- range .Values.ingress.additionalIngress }} +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ printf "%s-%s" $fullName .name }} + labels: + {{- $labels | nindent 4 }} + {{- with .annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .className }} + ingressClassName: {{ .className }} + {{- end }} + {{- if .tls }} + tls: + {{- range .tls }} + - hosts: + {{- range .hosts }} + - {{ . | quote }} + {{- end }} + secretName: {{ .secretName }} + {{- end }} + {{- end }} + rules: + {{- range .hosts }} + - host: {{ .host | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path }} + pathType: {{ .pathType }} + backend: + service: + name: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} + port: + number: {{ .backend.service.port.number }} + {{- end }} + {{- end }} + {{- end }} +{{- end }} {{- end }} diff --git a/stable/app/templates/migration-job.yaml b/stable/app/templates/migration-job.yaml index c0ecc0b6..f5cbb220 100644 --- a/stable/app/templates/migration-job.yaml +++ b/stable/app/templates/migration-job.yaml @@ -18,6 +18,9 @@ spec: metadata: labels: {{- include "app.selectorLabels" . | nindent 8 }} + {{- with .Values.podLabels }} + {{- toYaml . | nindent 8 }} + {{- end }} spec: restartPolicy: Never containers: diff --git a/stable/app/templates/otelcollector-configmap.yaml b/stable/app/templates/otelcollector-configmap.yaml new file mode 100644 index 00000000..2bed98eb --- /dev/null +++ b/stable/app/templates/otelcollector-configmap.yaml @@ -0,0 +1,11 @@ +{{- if .Values.otelcollector.enabled -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "app.fullname" . }}-otelcollector + labels: + {{- include "app.labels" . | nindent 4 }} +data: + config.yaml: |- +{{ .Values.otelcollector.config | indent 4 }} +{{- end }} diff --git a/stable/app/templates/service.yaml b/stable/app/templates/service.yaml index ec2a23e8..0f3e584b 100644 --- a/stable/app/templates/service.yaml +++ b/stable/app/templates/service.yaml @@ -1,9 +1,13 @@ +{{- if .Values.container.enabled }} +{{- $fullName := include "app.fullname" . -}} +{{- $labels := include "app.labels" . -}} +{{- $selectorLabels := include "app.selectorLabels" . -}} apiVersion: v1 kind: Service metadata: - name: {{ include "app.fullname" . }} + name: {{ $fullName }} labels: - {{- include "app.labels" . | nindent 4 }} + {{- $labels | nindent 4 }} {{- with .Values.service.annotations }} annotations: {{- toYaml . | nindent 4 }} @@ -18,4 +22,31 @@ spec: name: {{ $port.name }} {{- end }} selector: - {{- include "app.selectorLabels" . | nindent 4 }} + {{- $selectorLabels | nindent 4 }} +{{- if .Values.service.additionalService }} + {{- range .Values.service.additionalService }} +--- +apiVersion: v1 +kind: Service +metadata: + name: {{ printf "%s-%s" $fullName .name }} + labels: + {{- $labels | nindent 4 }} + {{- with .annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + type: {{ .type }} + ports: + {{- range $port := .ports }} + - port: {{ $port.port }} + targetPort: {{ $port.targetPort }} + protocol: {{ $port.protocol }} + name: {{ $port.name }} + {{- end }} + selector: + {{- $selectorLabels | nindent 4 }} + {{- end }} +{{- end }} +{{- end -}} \ No newline at end of file diff --git a/stable/app/values.yaml b/stable/app/values.yaml index 12f5d12f..4f743794 100644 --- a/stable/app/values.yaml +++ b/stable/app/values.yaml @@ -17,10 +17,15 @@ fullnameOverride: "" podAnnotations: {} +initContainers: {} + podSecurityContext: {} # -- Eg: fsGroup: 2000 -securityContext: {} +serviceAccountName: "" + +securityContext: + {} # -- capabilities: # -- drop: # -- - ALL @@ -45,7 +50,8 @@ service: ingress: enabled: false className: "" - annotations: {} + annotations: + {} # -- kubernetes.io/ingress.class: nginx # -- kubernetes.io/tls-acme: "true" hosts: @@ -63,7 +69,8 @@ ingress: # -- hosts: # -- - chart-example.local -resources: {} +resources: + {} # -- We usually recommend not to specify default resources and to leave this as a conscious # -- choice for the user. This also increases chances charts run on environments with little # -- resources, such as Minikube. If you do want to specify resources, uncomment the following @@ -93,8 +100,17 @@ cron: - app args: - startJob + serviceAccountName: "" + # resources: + # limits: + # cpu: "100m" + # memory: "100Mi" + # requests: + # cpu: "100m" + # memory: "100Mi" container: + enabled: true command: [] args: [] ports: @@ -122,9 +138,22 @@ container: # path: auth.json # secretName: app-service-account +hpa: + enabled: false + targetUtilization: + cpu: 70 + memory: 70 + stabilizationWindowSeconds: + scaleDown: 300 + scaleUp: 0 + replicas: + min: 1 + max: 1 + config: {} -secretConfig: {} +secretConfig: + {} # -- Example: database connection details can go under secret config # DB_HOST: localhost # DB_NAME: app @@ -141,3 +170,47 @@ telegraf: args: command: - telegraf + +otelcollector: + enabled: false + image: otel/opentelemetry-collector-contrib:0.89.0 + useEnvFromConfig: false + readinessProbe: + initialDelaySeconds: 15 + periodSeconds: 5 + livenessProbe: + initialDelaySeconds: 15 + periodSeconds: 5 + config: | + receivers: + otlp: + protocols: + grpc: + http: + processors: + exporters: + logging: + service: + pipelines: + traces: + receivers: [otlp] + processors: [] + exporters: [logging] + +podLabels: {} + +iap: + enabled: false + image: gojektech/iap-auth:0.1.0-99f7459df4481772d30176e3b39b3857cd538745 + containerPort: 8989 + refreshTimeSeconds: 3590s + serviceAccountMountPath: /etc/iap_auth + serviceAccountFileName: sa.json + serviceAccountSecret: iap-credentials-secret + logLevel: error + readinessProbe: + initialDelaySeconds: 0 + timeoutSeconds: 5 + livenessProbe: + initialDelaySeconds: 0 + timeoutSeconds: 5 diff --git a/stable/columbus/Chart.lock b/stable/columbus/Chart.lock deleted file mode 100644 index 524c225d..00000000 --- a/stable/columbus/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.2.0 -digest: sha256:2ff34dae9ff485fa30837e5777b90251e033662775ca37dcd30943472f1ea840 -generated: "2021-12-21T11:42:20.895938+07:00" diff --git a/stable/columbus/Chart.yaml b/stable/columbus/Chart.yaml deleted file mode 100644 index fa2090b4..00000000 --- a/stable/columbus/Chart.yaml +++ /dev/null @@ -1,10 +0,0 @@ -apiVersion: v2 -name: columbus -description: A Helm chart for Kubernetes -type: application -version: 0.1.2 -appVersion: "0.1.8" -dependencies: -- name: app - version: "0.2.0" - repository: "https://odpf.github.io/charts/" diff --git a/stable/columbus/README.md b/stable/columbus/README.md deleted file mode 100644 index 1e8267ad..00000000 --- a/stable/columbus/README.md +++ /dev/null @@ -1,92 +0,0 @@ -# Columbus - -A [helm](https://helm.sh/) chart for [Columbus](https://github.com/odpf/columbus) - -## Introduction - -This chart can be used for deploying [Columbus](https://github.com/odpf/columbus) on Kubernetes using helm - -## Prerequisites - -Install [helm3](https://helm.sh/docs/intro/install/#helm) - -## Installing the Chart - -### Install released version using Helm repository - -* Add the odpf charts repo - -```bash -$ helm repo add odpf https://odpf.github.io/charts/ -``` - -* Install it with Helm 3 - -```bash -$ helm install my-release odpf/columbus -``` - -### Install development version using master branch - -```bash -$ helm install --name my-release . -``` - -The command deploys Columbus on the Kubernetes cluster in the default configuration. The [configuration](#configuration) section lists the parameters that can be configured during installation. - -> **Tip**: List all releases using `helm list` - -## Uninstalling the Chart - -To uninstall/delete the `my-release` deployment: - -```bash -$ helm uninstall my-release -``` - -The command removes all the Kubernetes components associated with the chart and deletes the release. - -## Usage notes - -* Use init-columbus container in order to download and load proto jar files for Filter functionality -* Update flags for init-columbus and telegraf containers in order to use them, defaults are set to false -* Jolokia input for telegraf will be enabled only if jolokia is mentioned in JAVA_TOOL_OPTIONS in columbus config -* Sample Columbus configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/odpf/columbus/blob/main/docs/reference/configuration.md#configurations) -* Set the resource limits of containers according to the usage - -## Configuration - -The following table lists the configurable parameters of Columbus chart and their default values. - -## Values - -| Key | Type | Default | Description | -|-----|------|---------|-------------| -| app.config.SERVER_HOST | string | `localhost` | service host | -| app.config.SERVER_PORT | string | `3000` | service port | -| app.config.STATSD_ENABLED | string | `false` | enable statsd | -| app.config.STATSD_PREFIX | string | `columbus` | statsd prefix | -| app.config.LINEAGE_REFRESH_INTERVAL | string | `30m` | refresh interval for lineage building | -| app.config.NEW_RELIC_ENABLED | string | `false` | enable newrelic | -| app.config.NEW_RELIC_APP_NAME | string | `columbus` | newrelic app name | -| app.config.LOG_LEVEL | string | `info` | log level | -| app.secretConfig.ELASTICSEARCH_BROKERS | string | `` | elasticsearch brokers | -| app.secretConfig.STATSD_ADDRESS | string | `` | statsd address | -| app.secretConfig.NEW_RELIC_LICENSE_KEY | string | `` | newrelic license key | -| app.secretConfig.DB_HOST | string | `` | database host | -| app.secretConfig.DB_PORT | number | 5432 | database port | -| app.secretConfig.DB_NAME | string | `` | database name | -| app.secretConfig.DB_USER | string | `` | database user | -| app.secretConfig.DB_PASSWORD | string | `` | database password | -| app.secretConfig.DB_SSL_MODE | string | `disable` | database ssl mode | ---- - -Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. - -Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, - -```bash -$ helm install my-release -f values.yaml odpf/columbus -``` - -> **Tip**: You can use the default [values.yaml](values.yaml) diff --git a/stable/columbus/values.yaml b/stable/columbus/values.yaml deleted file mode 100644 index fbdbc0a4..00000000 --- a/stable/columbus/values.yaml +++ /dev/null @@ -1,60 +0,0 @@ -app: - image: - repository: odpf/columbus - pullPolicy: Always - tag: "0.1.8" - container: - command: - - columbus - args: - - serve - livenessProbe: - httpGet: - path: /ping - port: tcp - readinessProbe: - httpGet: - path: /ping - port: tcp - - migration: - enabled: true - command: - - columbus - args: - - migrate - - service: - annotations: - {} - - ingress: - enabled: true - annotations: - kubernetes.io/ingress.class: contour - hosts: - - host: columbus.example.com - paths: - - path: / - pathType: ImplementationSpecific - - config: - SERVER_PORT: 8080 - # SERVER_HOST: 0.0.0.0 - # STATSD_ENABLED: false - # STATSD_PREFIX: columbus - # LINEAGE_REFRESH_INTERVAL: 30m - # NEW_RELIC_ENABLED: false - # NEW_RELIC_APP_NAME: columbus - # LOG_LEVEL: info - - secretConfig: {} - # ELASTICSEARCH_BROKERS: ~ - # STATSD_ADDRESS: ~ - # NEW_RELIC_LICENSE_KEY: ~ - # DB_HOST: ~ - # DB_PORT: 5432 - # DB_NAME: ~ - # DB_USER: ~ - # DB_PASSWORD: ~ - # DB_SSL_MODE: disable diff --git a/stable/compass/Chart.lock b/stable/compass/Chart.lock deleted file mode 100644 index b63f36b6..00000000 --- a/stable/compass/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.4.2 -digest: sha256:5f0def8abcf2a1a197cbb9a63e06fb5076e34635b398011ce253dc84b5af3f75 -generated: "2022-10-27T16:42:43.502785+07:00" diff --git a/stable/compass/Chart.yaml b/stable/compass/Chart.yaml index 097d3f25..80ea09c8 100644 --- a/stable/compass/Chart.yaml +++ b/stable/compass/Chart.yaml @@ -2,9 +2,9 @@ apiVersion: v2 name: compass description: A Helm chart for running Compass in Kubernetes type: application -version: 0.1.5 -appVersion: "0.3.0" +version: 0.2.3 +appVersion: "0.6.5" dependencies: - name: app - version: "0.4.2" - repository: "https://odpf.github.io/charts/" + version: "0.6.0" + repository: "https://goto.github.io/charts/" diff --git a/stable/compass/README.md b/stable/compass/README.md index 11c9bb61..665f9434 100644 --- a/stable/compass/README.md +++ b/stable/compass/README.md @@ -1,10 +1,10 @@ # Compass -A [helm](https://helm.sh/) chart for [Compass](https://github.com/odpf/compass) +A [helm](https://helm.sh/) chart for [Compass](https://github.com/goto/compass) ## Introduction -This chart can be used for deploying [Compass](https://github.com/odpf/compass) on Kubernetes using helm +This chart can be used for deploying [Compass](https://github.com/goto/compass) on Kubernetes using helm ## Prerequisites @@ -14,16 +14,16 @@ Install [helm3](https://helm.sh/docs/intro/install/#helm) ### Install released version using Helm repository -* Add the odpf charts repo +* Add the gotocompany charts repo ```bash -$ helm repo add odpf https://odpf.github.io/charts/ +$ helm repo add gotocompany https://goto.github.io/charts/ ``` * Install it with Helm 3 ```bash -$ helm install my-release odpf/compass +$ helm install my-release gotocompany/compass ``` ### Install development version using master branch @@ -51,7 +51,7 @@ The command removes all the Kubernetes components associated with the chart and * Use init-compass container in order to download and load proto jar files for Filter functionality * Update flags for init-compass and telegraf containers in order to use them, defaults are set to false * Jolokia input for telegraf will be enabled only if jolokia is mentioned in JAVA_TOOL_OPTIONS in compass config -* Sample Compass configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/odpf/compass/blob/main/docs/reference/configuration.md#configurations) +* Sample Compass configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/goto/compass/blob/main/docs/reference/configuration.md#configurations) * Set the resource limits of containers according to the usage ## Configuration @@ -86,7 +86,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash -$ helm install my-release -f values.yaml odpf/compass +$ helm install my-release -f values.yaml gotocompany/compass ``` > **Tip**: You can use the default [values.yaml](values.yaml) diff --git a/stable/compass/values.yaml b/stable/compass/values.yaml index 29c1f40a..39555a64 100644 --- a/stable/compass/values.yaml +++ b/stable/compass/values.yaml @@ -1,13 +1,14 @@ app: image: - repository: odpf/compass + repository: gotocompany/compass pullPolicy: Always - tag: "0.3.0" + tag: "0.5.7" container: command: - compass args: - - serve + - server + - start livenessProbe: httpGet: path: /ping @@ -22,6 +23,7 @@ app: command: - compass args: + - server - migrate service: @@ -48,14 +50,18 @@ app: # COMPASS_SERVICE_HOST: 0.0.0.0 # COMPASS_STATSD_ENABLED: false # COMPASS_STATSD_PREFIX: compass - # COMPASS_NEWRELIC_ENABLED: false - # COMPASS_NEWRELIC_APPNAME: compass # COMPASS_LOG_LEVEL: info + # COMPASS_TELEMETRY_APP_NAME: "compass" + # COMPASS_TELEMETRY_NEWRELIC_ENABLED: false + # COMPASS_TELEMETRY_OPEN_TELEMETRY_ENABLED: false + # COMPASS_TELEMETRY_OPEN_TELEMETRY_COLLECTOR_ADDR: "127.0.0.1:4317" + # COMPASS_TELEMETRY_OPEN_TELEMETRY_PERIODIC_READ_INTERVAL: 5s + # COMPASS_TELEMETRY_OPEN_TELEMETRY_TRACE_SAMPLE_PROBABILITY: 0.1 secretConfig: {} # COMPASS_ELASTICSEARCH_BROKERS: ~ # COMPASS_STATSD_ADDRESS: ~ - # COMPASS_NEWRELIC_LICENSEKEY: ~ + # COMPASS_TELEMETRY_NEWRELIC_LICENSEKEY: ~ # COMPASS_DB_HOST: ~ # COMPASS_DB_PORT: 5432 # COMPASS_DB_NAME: ~ diff --git a/stable/consul/Chart.yaml b/stable/consul/Chart.yaml index 906d3611..dfeb658c 100644 --- a/stable/consul/Chart.yaml +++ b/stable/consul/Chart.yaml @@ -4,4 +4,4 @@ description: Minimal Helm chart for deploying consul server on kubernetes. name: consul version: 0.1.0 sources: -- https://github.com/odpf/charts +- https://github.com/goto/charts diff --git a/stable/consul/README.md b/stable/consul/README.md index 0a4eeabc..cbc23d97 100644 --- a/stable/consul/README.md +++ b/stable/consul/README.md @@ -6,7 +6,7 @@ Minimal Helm chart for deploying consul server on kubernetes. ## Source Code -- +- ## Values diff --git a/stable/columbus/.helmignore b/stable/dagger/.helmignore similarity index 100% rename from stable/columbus/.helmignore rename to stable/dagger/.helmignore diff --git a/stable/dagger/Chart.yaml b/stable/dagger/Chart.yaml new file mode 100644 index 00000000..10ee84f5 --- /dev/null +++ b/stable/dagger/Chart.yaml @@ -0,0 +1,6 @@ +apiVersion: v2 +name: dagger-deployment-chart +description: A Helm chart for dagger deployment +type: application +version: 0.9.0 +appVersion: "1.0" diff --git a/stable/dagger/templates/NOTES.txt b/stable/dagger/templates/NOTES.txt new file mode 100644 index 00000000..5563e76b --- /dev/null +++ b/stable/dagger/templates/NOTES.txt @@ -0,0 +1 @@ +Thank you for installing {{ .Chart.Name }}. \ No newline at end of file diff --git a/stable/dagger/templates/dagger.yaml b/stable/dagger/templates/dagger.yaml new file mode 100644 index 00000000..e2f1c436 --- /dev/null +++ b/stable/dagger/templates/dagger.yaml @@ -0,0 +1,163 @@ +apiVersion: flink.apache.org/v1beta1 +kind: FlinkDeployment +metadata: + name: {{ .Values.deployment_id }} + namespace: {{ .Values.namespace }} +spec: + image: {{ .Values.image }} + flinkVersion: v1_14 + flinkConfiguration: + blob.server.port: "6124" + taskmanager.rpc.port: "6122" + taskmanager.numberOfTaskSlots: "1" + metrics.latency.interval: "0" + web.timeout: "60000" + akka.ask.timeout: 10 min + rest.server.numThreads: "20" + slot.request.timeout: "300000" + cluster.evenly-spread-out-slots: "false" + rest.flamegraph.enabled: "false" + taskmanager.memory.managed.fraction: "0.2" + taskmanager.memory.managed.consumer-weights: OPERATOR:70,STATE_BACKEND:70,PYTHON:30 + classloader.resolve-order: parent-first + + #HA CONFIGS + state.backend: filesystem + state.backend.fs.checkpointdir: {{ .Values.dagger_checkpoint_url }}/{{ .Values.name }} + state.savepoints.dir: {{ .Values.dagger_savepoint_url }}/{{ .Values.name }} + blob.storage.directory: /tmp/flink-blobs + jobmanager.web.tmpdir: /tmp/flink-web + web.upload.dir: /tmp/flink-web/flink-uploads + + high-availability: org.apache.flink.kubernetes.highavailability.KubernetesHaServicesFactory + high-availability.storageDir: {{ .Values.dagger_k8s_ha_url }} + + fs.cosn.credentials.provider: org.apache.hadoop.fs.auth.EnvironmentVariableCredentialProvider + fs.cosn.bucket.region: ap-jakarta + + fs.oss.credentials.provider: com.aliyun.oss.common.auth.EnvironmentVariableCredentialsProvider + fs.oss.endpoint: oss-ap-southeast-5.aliyuncs.com + + metrics.reporters: stsd + metrics.reporter.stsd.class: com.gojek.de.dagger.metrics.StatsDReporter + metrics.reporter.stsd.host: localhost + metrics.reporter.stsd.port: "8125" + metrics.reporter.stsd.blacklistedTags: task_name,operator_name,task_attempt_id,metric_type + metrics.reporter.stsd.blacklistedMetrics: .*KafkaConsumer_current_offsets_.*,.*KafkaConsumer_committed_offsets.*,.*Shuffle_Netty.* + metrics.reporter.stsd.interval: 5 SECONDS + akka.framesize: 10485760b + client.timeout: 10 min + metrics.scope.jm: jobmanager.godata-id-flink-operator + metrics.scope.jm.job: jobmanager-job.godata-id-flink-operator.. + metrics.scope.tm: taskmanager.godata-id-flink-operator. + metrics.scope.tm.job: taskmanager-job.godata-id-flink-operator.. + metrics.scope.task: task.godata-id-flink-operator.... + metrics.scope.operator: operator.godata-id-flink-operator... + serviceAccount: flink + podTemplate: + apiVersion: v1 + kind: Pod + metadata: + name: common-pod-template + labels: + application: dagger + app-id: 02408523-06e0-4b4a-a3f4-8df6d429e21e + instance-id: {{ .Values.deployment_id }} + {{- range $key, $value := .Values.extra_labels }} + {{ $key }}: '{{ $value }}' + {{- end }} + spec: + serviceAccount: flink + containers: + - name: flink-main-container + {{- if or (eq .Values.cloud_provider "ali") (eq .Values.cloud_provider "tcc") }} + envFrom: + - secretRef: + name: dagger-user-credential + {{- end}} + {{- if eq .Values.cloud_provider "gcp" }} + env: + - name: GOOGLE_APPLICATION_CREDENTIALS + value: /var/secrets/google/gcp_key.json + {{- end}} + volumeMounts: + - mountPath: /etc/hadoop/conf + name: hadoop-config + {{- if eq .Values.cloud_provider "gcp" }} + - mountPath: /var/secrets/google + name: google-cloud-key + {{- end}} + - name: telegrafd + image: telegraf:1.18.3-alpine + imagePullPolicy: IfNotPresent + env: + - name: CLUSTER_NAME + value: {{ .Values.flink_name }} + - name: PROMETHEUS_URL + value: {{ .Values.prometheus_url }} + - name: DAGGER_URN + value: {{ .Values.urn }} + - name: TEAM_NAME + value: {{.Values.team }} + terminationMessagePath: /dev/termination-log + terminationMessagePolicy: File + resources: + limits: + cpu: 100m + memory: 100Mi + requests: + cpu: 100m + memory: 100Mi + volumeMounts: + - mountPath: /etc/telegraf + name: telegraf-config + volumes: + - name: hadoop-config + configMap: + defaultMode: 420 + name: flink-operator-hadoop-config + - name: telegraf-config + configMap: + defaultMode: 420 + name: flink-operator-telegraf-config + {{- if eq .Values.cloud_provider "gcp" }} + - name: google-cloud-key + secret: + defaultMode: 420 + secretName: flink-operator-gcp + {{- end}} + jobManager: + resource: + cpu: {{ .Values.resources.jobmanager.cpu }} + memory: {{ .Values.resources.jobmanager.memory }} + taskManager: + resource: + cpu: {{ .Values.resources.taskmanager.cpu }} + memory: {{ .Values.resources.taskmanager.memory }} + job: + jarURI: {{ .Values.jarURI }} + parallelism: {{ .Values.configuration.FLINK_PARALLELISM }} + upgradeMode: stateless + state: {{ .Values.state }} + entryClass: com.gotocompany.dagger.core.KafkaProtoSQLProcessor + savepointTriggerNonce: {{ .Values.savepointTriggerNonce }} + args: + {{ range .Values.programArgs }} + - {{ . }} + {{ end }} + logConfiguration: + "log4j-console.properties": | + rootLogger.level=INFO + rootLogger.appenderRef.console.ref=ConsoleAppender + logger.flink.name=org.apache.flink + logger.flink.level=INFO + logger.akka.name=akka + logger.akka.level=INFO + logger.kafka.name=org.apache.kafka + logger.kafka.level=INFO + logger.hadoop.name=org.apache.hadoop + logger.hadoop.level=INFO + appender.console.name=ConsoleAppender + appender.console.type=CONSOLE + appender.console.layout.type=PatternLayout + appender.console.layout.pattern=%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p %-60c - %m%n diff --git a/stable/dagger/values.yaml b/stable/dagger/values.yaml new file mode 100644 index 00000000..9ca8b567 --- /dev/null +++ b/stable/dagger/values.yaml @@ -0,0 +1,26 @@ +deployment_id: "dagger-test-deployment" +image: "dagger-flink-operator-image" +projectID: "g-pilotdata-gl" +name: "dagger-test" +team: "streaming" +flink_name: "flink-cluster" +prometheus_url: "http://prometheus:9090" +storage_protocol: gs +cloud_provider: gcp +resources: + jobmanager: + cpu: "1" + memory: "1024m" + taskmanager: + cpu: "1" + memory: "1024m" +jarURI: "dagger-minimal-jar" +configuration: + FLINK_PARALLELISM: 1 +state: "running" +programArgs: + - "--encodedArgs" + - WyItLVNPVVJDRV9LQUZLQV9DT05TVU1FX0xBUkdFX01FU1NBR0VfRU5BQkxFIixmYWxzZSwiLS1FTkFCTEVfU1RFTkNJTF9VUkwiLHRydWUsIi0tRkxJTktfSk9CX0lEIiwiZy1waWxvdGRhdGEtZ2wtdGVzdC1kYWdnZXIiLCItLVNJTktfSU5GTFVYX0JBVENIX1NJWkUiLDEwMCwiLS1TSU5LX0lORkxVWF9EQl9OQU1FIiwiIiwiLS1TSU5LX0lORkxVWF9GTFVTSF9EVVJBVElPTl9NUyIsMTAwMCwiLS1TSU5LX0lORkxVWF9NRUFTVVJFTUVOVF9OQU1FIiwiIiwiLS1TSU5LX0lORkxVWF9QQVNTV09SRCIsInJvb3QiLCItLVNJTktfSU5GTFVYX1JFVEVOVElPTl9QT0xJQ1kiLCJhdXRvZ2VuIiwiLS1TSU5LX0lORkxVWF9VUkwiLCIiLCItLVNJTktfSU5GTFVYX1VTRVJOQU1FIiwicm9vdCIsIi0tUFJPQ0VTU09SX0xPTkdCT1dfR0NQX0lOU1RBTkNFX0lEIiwidGVzdCIsIi0tUFJPQ0VTU09SX0xPTkdCT1dfR0NQX1BST0pFQ1RfSUQiLCJ0ZXN0IiwiLS1TSU5LX0tBRktBX0JST0tFUlMiLCIiLCItLVNJTktfS0FGS0FfVE9QSUMiLCIiLCItLU9VVFBVVF9QUk9UT19DTEFTU19QUkVGSVgiLCIiLCItLVNJTktfS0FGS0FfUFJPVE9fS0VZIiwiIiwiLS1TSU5LX0tBRktBX1BST1RPX01FU1NBR0UiLCIiLCItLVNJTktfS0FGS0FfU1RSRUFNIiwiIiwiLS1GTElOS19QQVJBTExFTElTTSIsMSwiLS1QT1JUQUxfVkVSU0lPTiIsIjIiLCItLVBST0NFU1NPUl9QUkVQUk9DRVNTT1JfQ09ORklHIiwiIiwiLS1QUk9DRVNTT1JfUFJFUFJPQ0VTU09SX0VOQUJMRSIsZmFsc2UsIi0tUFJPQ0VTU09SX1BPU1RQUk9DRVNTT1JfQ09ORklHIiwiIiwiLS1QUk9DRVNTT1JfUE9TVFBST0NFU1NPUl9FTkFCTEUiLGZhbHNlLCItLVNJTktfS0FGS0FfUFJPRFVDRV9MQVJHRV9NRVNTQUdFX0VOQUJMRSIsZmFsc2UsIi0tUkVESVNfU0VSVkVSIiwiIiwiLS1GTElOS19ST1dUSU1FX0FUVFJJQlVURV9OQU1FIiwicm93dGltZSIsIi0tU0lOS19UWVBFIiwiaW5mbHV4IiwiLS1GTElOS19TUUxfUVVFUlkiLCJTRUxFQ1QgKiBGUk9NIGRhdGFfc3RyZWFtc18wIiwiLS1TQ0hFTUFfUkVHSVNUUllfU1RFTkNJTF9FTkFCTEUiLHRydWUsIi0tU0NIRU1BX1JFR0lTVFJZX1NURU5DSUxfVVJMUyIsIiIsIi0tU1RSRUFNUyIsIlt7XCJJTlBVVF9TQ0hFTUFfRVZFTlRfVElNRVNUQU1QX0ZJRUxEX0lOREVYXCI6XCI1XCIsXCJJTlBVVF9TQ0hFTUFfUFJPVE9fQ0xBU1NcIjpcImdvamVrLmVzYi5ib29raW5nLkJvb2tpbmdMb2dNZXNzYWdlXCIsXCJJTlBVVF9TQ0hFTUFfVEFCTEVcIjpcImRhdGFfc3RyZWFtc18wXCIsXCJTT1VSQ0VfREVUQUlMU1wiOlt7XCJTT1VSQ0VfTkFNRVwiOlwiS0FGS0FfQ09OU1VNRVJcIixcIlNPVVJDRV9UWVBFXCI6XCJVTkJPVU5ERURcIn1dLFwiU09VUkNFX0tBRktBX0NPTlNVTUVSX0NPTkZJR19BVVRPX0NPTU1JVF9FTkFCTEVcIjpcImZhbHNlXCIsXCJTT1VSQ0VfS0FGS0FfQ09OU1VNRVJfQ09ORklHX0FVVE9fT0ZGU0VUX1JFU0VUXCI6XCJsYXRlc3RcIixcIlNPVVJDRV9LQUZLQV9DT05TVU1FUl9DT05GSUdfQk9PVFNUUkFQX1NFUlZFUlNcIjpcIlwiLFwiU09VUkNFX0tBRktBX0NPTlNVTUVSX0NPTkZJR19HUk9VUF9JRFwiOlwiXCIsXCJTT1VSQ0VfS0FGS0FfTkFNRVwiOlwiXCIsXCJTT1VSQ0VfS0FGS0FfVE9QSUNfTkFNRVNcIjpcIlwiLFwiU09VUkNFX1BBUlFVRVRfRklMRV9EQVRFX1JBTkdFXCI6bnVsbCxcIlNPVVJDRV9QQVJRVUVUX0ZJTEVfUEFUSFNcIjpudWxsfV0iLCItLUZMSU5LX1dBVEVSTUFSS19ERUxBWV9NUyIsMTAwMCwiLS1GTElOS19XQVRFUk1BUktfSU5URVJWQUxfTVMiLDYwMDAwLCItLVVERl9EQVJUX0dDU19CVUNLRVRfSUQiLCJwLWdvZGF0YS1kYWdnZXJzLWRhcnRzLXN0b3JhZ2UiLCItLVVERl9EQVJUX0dDU19QUk9KRUNUX0lEIiwiZ29kYXRhLXByb2R1Y3Rpb24iLCItLUZVTkNUSU9OX0ZBQ1RPUllfQ0xBU1NFUyIsImNvbS5nb3RvY29tcGFueS5kYWdnZXIuZnVuY3Rpb25zLnVkZnMuZmFjdG9yaWVzLkZ1bmN0aW9uRmFjdG9yeSxjb20uZ29qZWsuZGUuZmx1ZC51ZGZzLmZhY3Rvcmllcy5GdW5jdGlvbkZhY3RvcnkiLCItLVBZVEhPTl9VREZfRU5BQkxFIixmYWxzZSwiLS1QWVRIT05fVURGX0NPTkZJRyIsIntcIlBZVEhPTl9GSUxFU1wiOiBcImdzOi8vZ29kYXRhLWRhZ2dlci9weXRob24vbWFzdGVyL2xhdGVzdC9weXRob25fdWRmcy56aXBcIixcIlBZVEhPTl9SRVFVSVJFTUVOVFNcIjogXCJnczovL2dvZGF0YS1kYWdnZXIvcHl0aG9uL21hc3Rlci9sYXRlc3QvcmVxdWlyZW1lbnRzLnR4dFwiLFwiUFlUSE9OX0FSQ0hJVkVTXCI6IFwiZ3M6Ly9nb2RhdGEtZGFnZ2VyL3B5dGhvbi9tYXN0ZXIvbGF0ZXN0L2RhdGEuemlwI2RhdGFcIixcIlBZVEhPTl9GTl9FWEVDVVRJT05fQVJST1dfQkFUQ0hfU0laRVwiOiBcIjEwMDAwXCIsXCJQWVRIT05fRk5fRVhFQ1VUSU9OX0JVTkRMRV9TSVpFXCI6IFwiMTAwMDAwXCIsXCJQWVRIT05fRk5fRVhFQ1VUSU9OX0JVTkRMRV9USU1FXCI6IFwiMTAwMFwifSIsIi0tU0lOS19CSUdRVUVSWV9HT09HTEVfQ0xPVURfUFJPSkVDVF9JRCIsIiIsIi0tU0lOS19CSUdRVUVSWV9UQUJMRV9OQU1FIiwiIiwiLS1TSU5LX0JJR1FVRVJZX0RBVEFTRVRfTEFCRUxTIiwiIiwiLS1TSU5LX0JJR1FVRVJZX1RBQkxFX0xBQkVMUyIsIiIsIi0tU0lOS19CSUdRVUVSWV9EQVRBU0VUX05BTUUiLCIiLCItLVNJTktfQklHUVVFUllfQ1JFREVOVElBTF9QQVRIIiwiL3Zhci9zZWNyZXRzL2dvb2dsZS9nY3Bfa2V5Lmpzb24iLCItLVNJTktfQklHUVVFUllfVEFCTEVfUEFSVElUSU9OSU5HX0VOQUJMRSIsZmFsc2UsIi0tU0lOS19CSUdRVUVSWV9UQUJMRV9QQVJUSVRJT05fS0VZIiwiIiwiLS1TSU5LX0JJR1FVRVJZX1JPV19JTlNFUlRfSURfRU5BQkxFIixmYWxzZSwiLS1TSU5LX0JJR1FVRVJZX0NMSUVOVF9SRUFEX1RJTUVPVVRfTVMiLC0xLCItLVNJTktfQklHUVVFUllfQ0xJRU5UX0NPTk5FQ1RfVElNRU9VVF9NUyIsLTEsIi0tU0lOS19CSUdRVUVSWV9UQUJMRV9QQVJUSVRJT05fRVhQSVJZX01TIiwtMSwiLS1TSU5LX0JJR1FVRVJZX0RBVEFTRVRfTE9DQVRJT04iLCJhc2lhLXNvdXRoZWFzdDEiLCItLVNJTktfQklHUVVFUllfTUVUQURBVEFfTkFNRVNQQUNFIiwiIiwiLS1TSU5LX0JJR1FVRVJZX0FERF9NRVRBREFUQV9FTkFCTEVEIixmYWxzZSwiLS1TSU5LX0JJR1FVRVJZX01FVEFEQVRBX0NPTFVNTlNfVFlQRVMiLCIiLCItLVNJTktfTUVUUklDU19BUFBMSUNBVElPTl9QUkVGSVgiLCJkYWdnZXJfIiwiLS1TSU5LX0JJR1FVRVJZX0JBVENIX1NJWkUiLCIiLCItLVNJTktfQ09OTkVDVE9SX1NDSEVNQV9QUk9UT19NRVNTQUdFX0NMQVNTIiwiIiwiLS1TSU5LX0NPTk5FQ1RPUl9TQ0hFTUFfUFJPVE9fS0VZX0NMQVNTIiwiIiwiLS1TSU5LX0NPTk5FQ1RPUl9TQ0hFTUFfREFUQV9UWVBFIiwiUFJPVE9CVUYiLCItLVNJTktfQ09OTkVDVE9SX1NDSEVNQV9NRVNTQUdFX01PREUiLCJMT0dfTUVTU0FHRSIsIi0tU0lOS19DT05ORUNUT1JfU0NIRU1BX1BST1RPX0FMTE9XX1VOS05PV05fRklFTERTX0VOQUJMRSIsZmFsc2UsIi0tU0lOS19FUlJPUl9UWVBFU19GT1JfRkFJTFVSRSIsIiIsIi0tU0lOS19CSUdRVUVSWV9UQUJMRV9DTFVTVEVSSU5HX0VOQUJMRSIsZmFsc2UsIi0tU0lOS19CSUdRVUVSWV9UQUJMRV9DTFVTVEVSSU5HX0tFWVMiLCIiLCItLVNDSEVNQV9SRUdJU1RSWV9TVEVOQ0lMX0NBQ0hFX0FVVE9fUkVGUkVTSCIsZmFsc2UsIi0tU0NIRU1BX1JFR0lTVFJZX1NURU5DSUxfUkVGUkVTSF9TVFJBVEVHWSIsIkxPTkdfUE9MTElORyIsIi0tU0NIRU1BX1JFR0lTVFJZX1NURU5DSUxfQ0FDSEVfVFRMX01TIiwiOTAwMDAwIiwiLS1TSU5LX0tBRktBX0xJTkdFUl9NUyIsIjAiLCItLVNDSEVNQV9SRUdJU1RSWV9TVEVOQ0lMX0ZFVENIX1JFVFJJRVMiLCI0IiwiLS1TQ0hFTUFfUkVHSVNUUllfU1RFTkNJTF9GRVRDSF9CQUNLT0ZGX01JTl9NUyIsIjYwMDAwIiwiLS1TQ0hFTUFfUkVHSVNUUllfU1RFTkNJTF9GRVRDSF9USU1FT1VUX01TIiwiMTAwMCJd +extra_labels: {} +savepointTriggerNonce: 1 +urn: "dagger:urn" diff --git a/stable/dex/Chart.lock b/stable/dex/Chart.lock deleted file mode 100644 index a3a8c65e..00000000 --- a/stable/dex/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.4.0 -digest: sha256:e2d509925ddf8581f7b2a65bf0a37187ed8fde3a50d3e386a442ad34c60f8795 -generated: "2022-09-19T10:35:48.90955+05:30" diff --git a/stable/dex/Chart.yaml b/stable/dex/Chart.yaml index 3039ebb1..4dc139f1 100644 --- a/stable/dex/Chart.yaml +++ b/stable/dex/Chart.yaml @@ -2,9 +2,9 @@ apiVersion: v2 name: dex description: A Helm chart for Kubernetes type: application -version: 0.1.1 -appVersion: 0.1.1 +version: 0.1.5 +appVersion: 0.7.11 dependencies: - name: app - version: "0.4.0" - repository: "https://odpf.github.io/charts/" + version: "0.6.7" + repository: "https://goto.github.io/charts/" diff --git a/stable/dex/README.md b/stable/dex/README.md index c758b19d..df644c1a 100644 --- a/stable/dex/README.md +++ b/stable/dex/README.md @@ -1,10 +1,10 @@ # Dex -A [helm](https://helm.sh/) chart for [Dex](https://github.com/odpf/dex) +A [helm](https://helm.sh/) chart for [Dex](https://github.com/goto/dex) ## Introduction -This chart can be used for deploying [Dex](https://github.com/odpf/dex) on Kubernetes using helm. +This chart can be used for deploying [Dex](https://github.com/goto/dex) on Kubernetes using helm. ## Prerequisites @@ -14,16 +14,16 @@ Install [helm3](https://helm.sh/docs/intro/install/#helm) ### Install released version using Helm repository -- Add the odpf charts repo +- Add the gotocompany charts repo ```bash -$ helm repo add odpf https://odpf.github.io/charts/ +$ helm repo add gotocompany https://goto.github.io/charts/ ``` - Install it with Helm 3 ```bash -$ helm install my-release odpf/dex +$ helm install my-release gotocompany/dex ``` ### Install development version using master branch @@ -48,7 +48,7 @@ The command removes all the Kubernetes components associated with the chart and ## Usage notes -- Sample Dex configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/odpf/dex/blob/main/dex.yaml) +- Sample Dex configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/goto/dex/blob/main/dex.yaml) - Set the resource limits of containers according to the usage ## Configuration @@ -74,7 +74,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash -$ helm install my-release -f values.yaml odpf/dex +$ helm install my-release -f values.yaml gotocompany/dex ``` > **Tip**: You can use the default [values.yaml](values.yaml) diff --git a/stable/dex/values.yaml b/stable/dex/values.yaml index af77c6ea..ea75d6fc 100644 --- a/stable/dex/values.yaml +++ b/stable/dex/values.yaml @@ -1,6 +1,6 @@ app: image: - repository: odpf/dex + repository: gotocompany/dex pullPolicy: Always tag: "latest" container: @@ -17,6 +17,9 @@ app: httpGet: path: /ping port: tcp + podLabels: {} + + serviceAccountName: "" migration: enabled: false diff --git a/stable/entropy/Chart.lock b/stable/entropy/Chart.lock deleted file mode 100644 index c4546e8f..00000000 --- a/stable/entropy/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.3.4 -digest: sha256:41e9a8f47c6ad186219e24592ea05bb3d556474fdd2b088240901b978da3a8d3 -generated: "2022-06-17T12:16:20.299783+05:30" diff --git a/stable/entropy/Chart.yaml b/stable/entropy/Chart.yaml index f1c0b319..729928a6 100644 --- a/stable/entropy/Chart.yaml +++ b/stable/entropy/Chart.yaml @@ -2,9 +2,16 @@ apiVersion: v2 name: entropy description: A Helm chart for Kubernetes type: application -version: 0.1.3 -appVersion: 0.1.0 +version: 0.2.0 +appVersion: 0.1.26 dependencies: - name: app - version: "0.3.4" - repository: "https://odpf.github.io/charts/" + version: "0.7.0" + repository: "https://goto.github.io/charts/" + alias: app + condition: app.enabled +- name: app + version: "0.7.0" + repository: "https://goto.github.io/charts/" + alias: worker + condition: worker.enabled \ No newline at end of file diff --git a/stable/entropy/README.md b/stable/entropy/README.md index 9269e41d..b3c51d48 100644 --- a/stable/entropy/README.md +++ b/stable/entropy/README.md @@ -1,10 +1,10 @@ # Entropy -A [helm](https://helm.sh/) chart for [Entropy](https://github.com/odpf/entropy) +A [helm](https://helm.sh/) chart for [Entropy](https://github.com/goto/entropy) ## Introduction -This chart can be used for deploying [Entropy](https://github.com/odpf/entropy) on Kubernetes using helm. +This chart can be used for deploying [Entropy](https://github.com/goto/entropy) on Kubernetes using helm. ## Prerequisites @@ -14,16 +14,16 @@ Install [helm3](https://helm.sh/docs/intro/install/#helm) ### Install released version using Helm repository -- Add the odpf charts repo +- Add the gotocompany charts repo ```bash -$ helm repo add odpf https://odpf.github.io/charts/ +$ helm repo add gotocompany https://goto.github.io/charts/ ``` - Install it with Helm 3 ```bash -$ helm install my-release odpf/entropy +$ helm install my-release gotocompany/entropy ``` ### Install development version using master branch @@ -48,7 +48,7 @@ The command removes all the Kubernetes components associated with the chart and ## Usage notes -- Sample Entropy configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/odpf/entropy/blob/main/docs/reference/configuration.md#configurations) +- Sample Entropy configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/goto/entropy/blob/main/docs/reference/configuration.md#configurations) - Set the resource limits of containers according to the usage ## Configuration @@ -77,7 +77,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash -$ helm install my-release -f values.yaml odpf/entropy +$ helm install my-release -f values.yaml gotocompany/entropy ``` > **Tip**: You can use the default [values.yaml](values.yaml) diff --git a/stable/entropy/values.yaml b/stable/entropy/values.yaml index 4bbd136a..a7d98f22 100644 --- a/stable/entropy/values.yaml +++ b/stable/entropy/values.yaml @@ -1,6 +1,6 @@ app: image: - repository: odpf/entropy + repository: gotocompany/entropy pullPolicy: Always tag: "latest" container: @@ -17,6 +17,8 @@ app: httpGet: path: /ping port: tcp + podLabels: {} + serviceAccountName: "" migration: enabled: true @@ -50,3 +52,47 @@ app: # NEW_RELIC_APP_NAME: entropy secretConfig: {} + +worker: + image: + repository: gotocompany/entropy + pullPolicy: Always + tag: "latest" + container: + command: + - entropy + args: + - worker + livenessProbe: + exec: + command: + - /bin/sh + - -c + - ps -ef | grep entropy | grep -v grep + initialDelaySeconds: 10 + periodSeconds: 10 + readinessProbe: + exec: + command: + - /bin/sh + - -c + - ps -ef | grep entropy | grep -v grep + initialDelaySeconds: 10 + periodSeconds: 10 + podLabels: {} + + migration: + enabled: false + + ingress: + enabled: false + + config: + LOG_LEVEL: INFO + SYNCER_SYNC_INTERVAL: 1s + SYNCER_REFRESH_INTERVAL: 3s + SYNCER_EXTEND_LOCK_BY: 5s + SYNCER_SYNC_BACKOFF_INTERVAL: 5s + SYNCER_MAX_RETRIES: 5 + + secretConfig: {} \ No newline at end of file diff --git a/stable/firehose/Chart.yaml b/stable/firehose/Chart.yaml index 8a4d71ed..d983a323 100644 --- a/stable/firehose/Chart.yaml +++ b/stable/firehose/Chart.yaml @@ -2,5 +2,5 @@ apiVersion: v2 name: firehose description: A Helm chart for deploying Firehose on Kubernetes type: application -version: 0.1.3 +version: 0.1.20 appVersion: 0.7.1 diff --git a/stable/firehose/README.md b/stable/firehose/README.md index bae2722f..be884b9f 100644 --- a/stable/firehose/README.md +++ b/stable/firehose/README.md @@ -1,10 +1,10 @@ # Firehose -A [helm](https://helm.sh/) chart for [Firehose](https://github.com/odpf/firehose) +A [helm](https://helm.sh/) chart for [Firehose](https://github.com/goto/firehose) ## Introduction -This chart can be used for deploying [Firehose](https://github.com/odpf/firehose) on Kubernetes using helm +This chart can be used for deploying [Firehose](https://github.com/goto/firehose) on Kubernetes using helm ## Prerequisites @@ -14,16 +14,16 @@ Install [helm3](https://helm.sh/docs/intro/install/#helm) ### Install released version using Helm repository -* Add the odpf charts repo +* Add the gotocompany charts repo ```bash -$ helm repo add odpf https://odpf.github.io/charts/ +$ helm repo add gotocompany https://goto.github.io/charts/ ``` * Install it with Helm 3 ```bash -$ helm install my-release odpf/firehose +$ helm install my-release gotocompany/firehose ``` ### Install development version using master branch @@ -51,7 +51,7 @@ The command removes all the Kubernetes components associated with the chart and * Use init-firehose container in order to download and load proto jar files for Filter functionality * Update flags for init-firehose and telegraf containers in order to use them, defaults are set to false * Jolokia input for telegraf will be enabled only if jolokia is mentioned in JAVA_TOOL_OPTIONS in firehose config -* Sample Firehose configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/odpf/firehose/blob/main/docs/reference/configuration.md#configurations) +* Sample Firehose configs are mentioned in values.yaml file, update them according to usecase, [read more](https://github.com/goto/firehose/blob/main/docs/reference/configuration.md#configurations) * Set the resource limits of containers according to the usage ## Configuration @@ -62,10 +62,10 @@ The following table lists the configurable parameters of Firehose chart and thei | Key | Type | Default | Description | |-----|------|---------|-------------| -| firehose.args | list | `['java', '-cp', 'bin/*:/work-dir/*', 'io.odpf.firehose.launch.Main', '-server', '-Dlogback.configurationFile=etc/firehose/logback.xml', '-Xloggc:/var/log/firehose']` | args for firehose container | -| firehose.config | object | `{"SOURCE_KAFKA_BROKERS": "127.0.0.1:6667", "SOURCE_KAFKA_CONSUMER_GROUP_ID": "kafka-consumer-group-id", "SOURCE_KAFKA_TOPIC": "sample-topic", "SINK_TYPE": "log", "SOURCE_KAFKA_CONSUMER_CONFIG_AUTO_OFFSET_RESET": "latest", "INPUT_SCHEMA_PROTO_CLASS": "com.github.firehose.sampleLogProto.SampleLogMessage", "JAVA_TOOL_OPTIONS": "-javaagent:jolokia-jvm-agent.jar=port=8778,host=localhost"}` | env variables required by firehose, [read more](https://github.com/odpf/firehose/blob/main/docs/reference/configuration.md#configurations) | +| firehose.args | list | `['java', '-cp', 'bin/*:/work-dir/*', 'com.gotocompany.firehose.launch.Main', '-server', '-Dlogback.configurationFile=etc/firehose/logback.xml', '-Xloggc:/var/log/firehose']` | args for firehose container | +| firehose.config | object | `{"SOURCE_KAFKA_BROKERS": "127.0.0.1:6667", "SOURCE_KAFKA_CONSUMER_GROUP_ID": "kafka-consumer-group-id", "SOURCE_KAFKA_TOPIC": "sample-topic", "SINK_TYPE": "log", "SOURCE_KAFKA_CONSUMER_CONFIG_AUTO_OFFSET_RESET": "latest", "INPUT_SCHEMA_PROTO_CLASS": "com.github.firehose.sampleLogProto.SampleLogMessage", "JAVA_TOOL_OPTIONS": "-javaagent:jolokia-jvm-agent.jar=port=8778,host=localhost"}` | env variables required by firehose, [read more](https://github.com/goto/firehose/blob/main/docs/reference/configuration.md#configurations) | | firehose.image.pullPolicy | string | `"IfNotPresent"` | the Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value for firehose container | -| firehose.image.repository | string | `"odpf/firehose"` | docker repository to download firehose image | +| firehose.image.repository | string | `"gotocompany/firehose"` | docker repository to download firehose image | | firehose.image.tag | string | `"1.1.0"` | firehose docker image tag | | firehose.resources.limits.cpu | string | `"200m"` | firehose container cpu limit | | firehose.resources.limits.memory | string | `"512Mi"` | firehose container memory limit | @@ -93,6 +93,30 @@ The following table lists the configurable parameters of Firehose chart and thei | telegraf.resources.limits.memory | string | `"64Mi"` | telegraf container memory limit | | telegraf.resources.requests.cpu | string | `"50m"` | telegraf container cpu requests | | telegraf.resources.requests.memory | string | `"64Mi"` | telegraf container memory requests | +| tolerations | list | - | List of Kubernetes [tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) | +| volumes | list | - | List of Kubernetes [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) | +| volumeMounts | list | - | List of Kubernetes [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/#using-volumes) | +| nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution | list | - | List of Kubernetes [node affinity match expressions](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#node-affinity) that are required for the pod to be scheduled on a node | +| nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution | list | - | List of Kubernetes [preferred node affinity match expressions](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#preferred-during-scheduling-ignoreduringexecution) for the pod scheduling | +| tolerations.key | string | `"key1"` | Key to be mached | +| tolerations.operator | string | `"Equal"` | Operation to be checked | +| tolerations.value | string | `"value1"` | Values against which operation is performed | +| tolerations.effect | string | `"NoSchedule"` | Taint effect | +| volumes.name | string | - | Name of the Kubernetes volume | +| volumes.items.key | string | - | Key of the secret data | +| volumes.items.path | string | - | Path where the secret data will be mounted | +| volumes.secretName | string | - | Name of the Kubernetes secret | +| volumes.defaultMode | integer | - | Default file permissions for the volume | +| volumeMounts.name | string | - | Name of the Kubernetes volume | +| volumeMounts.mountPath | string | - | Path within the container where the volume should be mounted | +| nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution.key | string | - | Key of the node affinity match expression | +| nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution.operator | string | - | Operator of the node affinity match expression | +| nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution.values | list | - | List of values of the node affinity match expression | +| nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution.weight | integer | - | Weight of the preferred node affinity match expression | +| nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution.preference.key | string | - | Key of the preferred node affinity match expression | +| nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution.preference.operator | string | - | Operator of the preferred node affinity match expression | +| nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution.preference.values | list | - | List of values of the preferred node affinity match expression | + --- Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. @@ -100,7 +124,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash -$ helm install my-release -f values.yaml odpf/firehose +$ helm install my-release -f values.yaml gotocompany/firehose ``` > **Tip**: You can use the default [values.yaml](values.yaml) diff --git a/stable/firehose/templates/_helpers.tpl b/stable/firehose/templates/_helpers.tpl index deebc3ae..5f3bc8c0 100644 --- a/stable/firehose/templates/_helpers.tpl +++ b/stable/firehose/templates/_helpers.tpl @@ -17,7 +17,6 @@ {{- define "firehose.telegraf.conf" }} [global_tags] - app = "{{ include "firehose.fullname" . }}" {{- range $k, $v := .Values.telegraf.config.additional_global_tags }} {{ $k }} = "{{ $v }}" {{- end }} @@ -56,7 +55,7 @@ {{- end }} [[inputs.statsd]] allowed_pending_messages = 10000 - delete_counters = true + delete_counters = false delete_gauges = true delete_sets = true delete_timings = true diff --git a/stable/firehose/templates/configmap-firehose.yaml b/stable/firehose/templates/configmap-firehose.yaml index 91a7ed2e..2bd3cbb3 100644 --- a/stable/firehose/templates/configmap-firehose.yaml +++ b/stable/firehose/templates/configmap-firehose.yaml @@ -1,7 +1,7 @@ apiVersion: v1 kind: ConfigMap metadata: - name: {{ include "firehose.fullname" . }}-config + name: {{ include "firehose.fullname" . }} namespace: {{ .Release.Namespace }} labels: {{- range $key, $value := .Values.labels }} {{ $key }}: {{ $value }} diff --git a/stable/firehose/templates/configmap-telegraf.yaml b/stable/firehose/templates/configmap-telegraf.yaml index c2e0a711..3d8a1e12 100644 --- a/stable/firehose/templates/configmap-telegraf.yaml +++ b/stable/firehose/templates/configmap-telegraf.yaml @@ -2,7 +2,7 @@ apiVersion: v1 kind: ConfigMap metadata: - name: {{ include "firehose.fullname" . }}-telegraf-config + name: {{ include "firehose.fullname" . }}-telegraf namespace: {{ .Release.Namespace }} labels: {{- range $key, $value := .Values.labels }} {{ $key }}: {{ $value }} diff --git a/stable/firehose/templates/deployment.yaml b/stable/firehose/templates/deployment.yaml index 675c04f2..932a96fd 100644 --- a/stable/firehose/templates/deployment.yaml +++ b/stable/firehose/templates/deployment.yaml @@ -29,16 +29,24 @@ spec: name: {{ .Chart.Name }} image: "{{ .Values.firehose.image.repository }}:{{ .Values.firehose.image.tag | default .Chart.AppVersion }}" imagePullPolicy: {{ .Values.firehose.image.pullPolicy }} + {{- if or (index .Values "init-firehose" "enabled") (gt (len .Values.mountSecrets) 0 ) }} volumeMounts: {{- if index .Values "init-firehose" "enabled" }} - mountPath: /work-dir name: workdir {{- end }} + {{- if (gt (len .Values.mountSecrets) 0 ) }} + - mountPath: /etc/secret + name: {{ include "firehose.fullname" . }}-mount + {{- end }} + {{- end }} resources: {{- toYaml .Values.firehose.resources | nindent 12 }} envFrom: - configMapRef: - name: {{ include "firehose.fullname" . }}-config + name: {{ include "firehose.fullname" . }} + - secretRef: + name: {{ include "firehose.fullname" . }} env: - name: POD_NAME valueFrom: @@ -75,13 +83,63 @@ spec: - mountPath: /work-dir name: workdir {{- end }} + {{if or (index .Values.telegraf.enabled) (index .Values "init-firehose" "enabled") (gt (len .Values.mountSecrets) 0 ) }} volumes: {{- if index .Values.telegraf.enabled }} - configMap: - name: {{ include "firehose.fullname" . }}-telegraf-config + name: {{ include "firehose.fullname" . }}-telegraf name: telegraf-conf {{- end }} {{- if index .Values "init-firehose" "enabled" }} - emptyDir: {} name: workdir {{- end }} + {{- if (gt (len .Values.mountSecrets) 0 ) }} + - name: {{ include "firehose.fullname" . }}-mount + secret: + secretName: {{ include "firehose.fullname" . }}-mount + defaultMode: 420 + items: + {{- range $_, $mountSecret := .Values.mountSecrets}} + - key: {{$mountSecret.key}} + path: {{$mountSecret.path}} + {{- end}} + {{- end}} + {{- end}} + + {{- if (gt (len .Values.tolerations) 0) }} + tolerations: {{- range $_, $toleration := .Values.tolerations }} + - key: {{ $toleration.key }} + operator: {{ $toleration.operator }} + value: {{ $toleration.value }} + effect: {{ $toleration.effect }} + {{- end }} + {{- end}} + {{- if or (gt (len .Values.nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution) 0) (gt (len .Values.nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution) 0) }} + affinity: + nodeAffinity: + {{- if (gt (len .Values.nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution) 0) }} + requiredDuringSchedulingIgnoredDuringExecution: + nodeSelectorTerms: + - matchExpressions: {{- range $_, $expression := .Values.nodeAffinityMatchExpressions.requiredDuringSchedulingIgnoredDuringExecution }} + - key: {{ $expression.key }} + operator: {{ $expression.operator }} + values: {{- range $expression.values }} + - "{{ . }}" + {{- end}} + {{- end}} + {{- end}} + {{- if (gt (len .Values.nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution) 0) }} + preferredDuringSchedulingIgnoredDuringExecution: {{- range $_, $expression := .Values.nodeAffinityMatchExpressions.preferredDuringSchedulingIgnoredDuringExecution }} + - weight: {{ $expression.weight}} + preference: + matchExpressions: {{- range $_, $pref := $expression.preference }} + - key: {{ $pref.key }} + operator: {{ $pref.operator }} + values: {{- range $pref.values }} + - {{ . }} + {{- end}} + {{- end}} + {{- end}} + {{- end}} + {{- end}} \ No newline at end of file diff --git a/stable/firehose/templates/secrets.yaml b/stable/firehose/templates/secrets.yaml new file mode 100644 index 00000000..cbede9ae --- /dev/null +++ b/stable/firehose/templates/secrets.yaml @@ -0,0 +1,25 @@ +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "firehose.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: {{- range $key, $value := .Values.labels }} + {{ $key }}: {{ $value }} + {{- end }} +stringData: + {{- toYaml .Values.envSecrets | nindent 2 }} + +--- + +apiVersion: v1 +kind: Secret +metadata: + name: {{ include "firehose.fullname" . }}-mount + namespace: {{ .Release.Namespace }} + labels: {{- range $key, $value := .Values.labels }} + {{ $key }}: {{ $value }} + {{- end }} +data: + {{- range $_, $mountSecret := .Values.mountSecrets }} + {{$mountSecret.key }}: {{ $mountSecret.value }} + {{- end }} diff --git a/stable/firehose/values.yaml b/stable/firehose/values.yaml index a09dadc4..0cba5d8b 100644 --- a/stable/firehose/values.yaml +++ b/stable/firehose/values.yaml @@ -9,7 +9,7 @@ labels: { "application": "firehose" } firehose: image: - repository: odpf/firehose + repository: gotocompany/firehose pullPolicy: IfNotPresent tag: 1.1.0 config: @@ -20,12 +20,14 @@ firehose: SOURCE_KAFKA_CONSUMER_CONFIG_AUTO_OFFSET_RESET: latest INPUT_SCHEMA_PROTO_CLASS: com.github.firehose.sampleLogProto.SampleLogMessage JAVA_TOOL_OPTIONS: "-javaagent:jolokia-jvm-agent.jar=port=8778,host=localhost" + METRIC_STATSD_HOST: localhost + METRIC_STATSD_PORT: "8125" args: [ "java", "-cp", "bin/*:/work-dir/*", - "io.odpf.firehose.launch.Main", + "com.gotocompany.firehose.launch.Main", "-server", "-Dlogback.configurationFile=etc/firehose/logback.xml", "-Xloggc:/var/log/firehose", @@ -38,6 +40,12 @@ firehose: cpu: 200m memory: 512Mi +tolerations: [] + +nodeAffinityMatchExpressions: + requiredDuringSchedulingIgnoredDuringExecution: [] + preferredDuringSchedulingIgnoredDuringExecution: [] + init-firehose: enabled: false image: @@ -71,8 +79,12 @@ telegraf: # authorization: Bearer resources: limits: - cpu: 50m - memory: 64Mi + cpu: 600m + memory: 200Mi requests: - cpu: 50m - memory: 64Mi + cpu: 200m + memory: 100Mi + +envSecrets: {} +mountSecrets: [] + diff --git a/stable/guardian/Chart.lock b/stable/guardian/Chart.lock deleted file mode 100644 index dcb1889a..00000000 --- a/stable/guardian/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.4.0 -digest: sha256:e2d509925ddf8581f7b2a65bf0a37187ed8fde3a50d3e386a442ad34c60f8795 -generated: "2022-08-07T00:34:15.181033+05:30" diff --git a/stable/guardian/Chart.yaml b/stable/guardian/Chart.yaml index a114dec4..bcfe9f6b 100644 --- a/stable/guardian/Chart.yaml +++ b/stable/guardian/Chart.yaml @@ -15,7 +15,7 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.2.0 +version: 0.2.9 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to @@ -25,5 +25,5 @@ appVersion: "0.3.0" dependencies: - name: app - version: "0.4.0" - repository: "https://odpf.github.io/charts/" + version: "0.6.9" + repository: "https://goto.github.io/charts/" diff --git a/stable/guardian/README.md b/stable/guardian/README.md index 1b38ed66..0694f419 100644 --- a/stable/guardian/README.md +++ b/stable/guardian/README.md @@ -8,7 +8,7 @@ A Helm chart for Kubernetes | Repository | Name | Version | |------------|------|---------| -| https://odpf.github.io/charts/ | app | 0.4.0 | +| https://goto.github.io/charts/ | app | 0.4.0 | ## Values @@ -53,7 +53,7 @@ A Helm chart for Kubernetes | app.cron.jobs[2].schedule | string | `"*/20 * * * *"` | | | app.fullnameOverride | string | `""` | | | app.image.pullPolicy | string | `"Always"` | | -| app.image.repository | string | `"odpf/guardian"` | | +| app.image.repository | string | `"gotocompany/guardian"` | | | app.image.tag | string | `"latest"` | | | app.ingress.annotations."kubernetes.io/ingress.class" | string | `"contour"` | | | app.ingress.enabled | bool | `true` | | diff --git a/stable/guardian/values.yaml b/stable/guardian/values.yaml index d169c19a..ba2a42c0 100644 --- a/stable/guardian/values.yaml +++ b/stable/guardian/values.yaml @@ -5,8 +5,10 @@ app: ## Value to fully override guardian.fullname template fullnameOverride: "" + podLabels: {} + image: - repository: odpf/guardian + repository: gotocompany/guardian pullPolicy: Always tag: latest container: @@ -38,24 +40,27 @@ app: - name: "fetch-resources" schedule: "0 */2 * * *" restartPolicy: Never - command: [] - args: + args: [] + command: + - ./guardian - job - run - fetch_resources - name: "appeal-expiration-reminder" schedule: "0 9 * * *" restartPolicy: Never - command: [] - args: + args: [] + command: + - ./guardian - job - run - appeal_expiration_reminder - name: "appeal-expiration-revocation" schedule: "*/20 * * * *" restartPolicy: Never - command: [] - args: + args: [] + command: + - ./guardian - job - run - appeal_expiration_revocation @@ -78,7 +83,6 @@ app: config: LOG_LEVEL: info AUTHENTICATED_USER_HEADER_KEY: x-authenticated-user-email - NOTIFIER_PROVIDER: slack JOBS_FETCH_RESOURCES_ENABLED: false JOBS_FETCH_RESOURCES_INTERVAL: "0 */2 * * *" JOBS_REVOKE_EXPIRED_ACCESS_ENABLED: false @@ -89,7 +93,6 @@ app: secretConfig: ENCRYPTION_SECRET_KEY: - NOTIFIER_ACCESS_TOKEN: DB_HOST: localhost DB_PORT: DB_NAME: guardian diff --git a/stable/metabase/Chart.yaml b/stable/metabase/Chart.yaml new file mode 100644 index 00000000..9c765109 --- /dev/null +++ b/stable/metabase/Chart.yaml @@ -0,0 +1,25 @@ +apiVersion: v2 +name: metabase +description: Metabase Helm chart including telegraf + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.2 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +# It is recommended to use it with quotes. +appVersion: "v0.50.10" + diff --git a/stable/metabase/README.md b/stable/metabase/README.md new file mode 100644 index 00000000..e5b8df9a --- /dev/null +++ b/stable/metabase/README.md @@ -0,0 +1,61 @@ +# metabase + +![Version: 0.1.0](https://img.shields.io/badge/Version-0.1.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: v0.50.10](https://img.shields.io/badge/AppVersion-v0.50.10-informational?style=flat-square) + +Metabase Helm chart including telegraf + +## Values + +| Key | Type | Default | Description | +|-----|------|---------|-------------| +| affinity | object | `{}` | | +| ingress.annotations | object | `{}` | | +| ingress.enabled | bool | `true` | | +| ingress.hosts | string | `nil` | | +| ingress.ingressClassName | string | `"contour"` | | +| ingress.labels | string | `nil` | | +| ingress.path | string | `"/"` | | +| ingress.tls | string | `nil` | | +| labels | object | `{}` | | +| metabase.database.dbname | string | `"db_name"` | | +| metabase.database.host | string | `"localhost"` | | +| metabase.database.password | string | `"db_password"` | | +| metabase.database.port | int | `5432` | | +| metabase.database.type | string | `"postgres"` | | +| metabase.database.username | string | `"db_username"` | | +| metabase.emojiLogging | bool | `true` | | +| metabase.image.pullPolicy | string | `"IfNotPresent"` | | +| metabase.image.repository | string | `"metabase/metabase"` | | +| metabase.image.tag | string | `"v0.50.10"` | | +| metabase.jetty | string | `nil` | | +| metabase.listen.host | string | `"0.0.0.0"` | | +| metabase.listen.port | int | `3000` | | +| metabase.livenessProbe.failureThreshold | int | `6` | | +| metabase.livenessProbe.initialDelaySeconds | int | `120` | | +| metabase.livenessProbe.path | string | `"/api/health"` | | +| metabase.livenessProbe.timeoutSeconds | int | `30` | | +| metabase.password.complexity | string | `"weak"` | | +| metabase.password.length | int | `6` | | +| metabase.readinessProbe.initialDelaySeconds | int | `30` | | +| metabase.readinessProbe.path | string | `"/api/health"` | | +| metabase.readinessProbe.periodSeconds | int | `5` | | +| metabase.readinessProbe.timeoutSeconds | int | `3` | | +| metabase.resources | object | `{}` | | +| metabase.ssl.enabled | bool | `false` | | +| metabase.timeZone | string | `"UTC"` | | +| nodeSelector | object | `{}` | | +| podAnnotations | object | `{}` | | +| replicaCount | int | `1` | | +| service.annotations | object | `{}` | | +| service.externalPort | int | `80` | | +| service.internalPort | int | `3000` | | +| service.nodePort | string | `nil` | | +| service.type | string | `"ClusterIP"` | | +| telegraf.image.pullPolicy | string | `"IfNotPresent"` | | +| telegraf.image.repository | string | `"telegraf"` | | +| telegraf.image.tag | string | `"1.19.3-alpine"` | | +| telegraf.resources | object | `{}` | | +| tolerations | list | `[]` | | + +---------------------------------------------- +Autogenerated from chart metadata using [helm-docs v1.14.2](https://github.com/norwoodj/helm-docs/releases/v1.14.2) diff --git a/stable/metabase/templates/NOTES.txt b/stable/metabase/templates/NOTES.txt new file mode 100644 index 00000000..88b49819 --- /dev/null +++ b/stable/metabase/templates/NOTES.txt @@ -0,0 +1,17 @@ +1. Get the application URL by running these commands: +{{- if .Values.ingress.hostname }} + http://{{- .Values.ingress.hostname }} +{{- else if contains "NodePort" .Values.service.type }} + export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "metabase.fullname" . }}) + export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}") + echo http://$NODE_IP:$NODE_PORT +{{- else if contains "LoadBalancer" .Values.service.type }} + NOTE: It may take a few minutes for the LoadBalancer IP to be available. + You can watch the status of by running 'kubectl get svc -w {{ template "metabase.fullname" . }}' + export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "metabase.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}') + echo http://$SERVICE_IP:{{ .Values.service.externalPort }} +{{- else if contains "ClusterIP" .Values.service.type }} + export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "metabase.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") + echo "Visit http://127.0.0.1:8080 to use your application" + kubectl port-forward --namespace {{ .Release.Namespace }} $POD_NAME 8080:{{ .Values.service.internalPort }} +{{- end }} diff --git a/stable/metabase/templates/_helper.tpl b/stable/metabase/templates/_helper.tpl new file mode 100644 index 00000000..59aa9917 --- /dev/null +++ b/stable/metabase/templates/_helper.tpl @@ -0,0 +1,37 @@ +{{/* vim: set filetype=mustache: */}} +{{/* +Expand the name of the chart. +*/}} +{{- define "metabase.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "metabase.fullname" -}} +{{- if .Values.fullnameOverride -}} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- $name := default .Chart.Name .Values.nameOverride -}} +{{- if contains $name .Release.Name -}} +{{- printf .Release.Name | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end -}} +{{- end -}} + + +{{/* +Return the apiVersion of deployment. +*/}} +{{- define "deployment.apiVersion" -}} +{{- if semverCompare "<1.14-0" .Capabilities.KubeVersion.GitVersion -}} +{{- print "extensions/v1beta1" -}} +{{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}} +{{- print "apps/v1" -}} +{{- end -}} +{{- end -}} diff --git a/stable/metabase/templates/config.yaml b/stable/metabase/templates/config.yaml new file mode 100644 index 00000000..41088d20 --- /dev/null +++ b/stable/metabase/templates/config.yaml @@ -0,0 +1,88 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "metabase.fullname" . }}-config + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} +data: + {{- if .Values.log4jProperties }} + log4j.properties: +{{ toYaml .Values.log4jProperties | indent 4}} + {{- end}} + +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "metabase.fullname" . }}-telegraf-config + namespace: {{ .Release.Namespace }} +data: + telegraf.conf: |- + [global_tags] + {{- range $key, $value := .Values.labels }} + {{ $key }} = {{ $value | quote }} + {{- end }} +{{ if .Values.telegraf.influx }} + [[outputs.influxdb]] + urls = [{{ .Values.telegraf.influx.address | quote }}] + database = {{ .Values.telegraf.influx.databaseName | quote }} + retention_policy = "" + write_consistency = "any" + timeout = "5s" +{{ end }} +{{ if .Values.telegraf.prometheusRemoteWrite }} + [[outputs.http]] + url = {{ .Values.telegraf.prometheusRemoteWrite.url | quote }} + data_format = "prometheusremotewrite" + [outputs.http.headers] + Content-Type= "application/x-protobuf" + Content-Encoding = "snappy" + X-Prometheus-Remote-Write-Version= "0.1.0" +{{ end }} + [[inputs.statsd]] + service_address = ":8125" + percentiles = [ 90.0, 95.0, 99.0] + + [[inputs.jolokia2_agent]] + urls = ["http://localhost:7777/jolokia"] + + [[inputs.jolokia2_agent.metric]] + name = "java_runtime" + mbean = "java.lang:type=Runtime" + paths = ["Uptime"] + + [[inputs.jolokia2_agent.metric]] + name = "java_memory" + mbean = "java.lang:type=Memory" + paths = ["HeapMemoryUsage", "NonHeapMemoryUsage", "ObjectPendingFinalizationCount"] + + [[inputs.jolokia2_agent.metric]] + name = "java_garbage_collector" + mbean = "java.lang:name=*,type=GarbageCollector" + paths = ["CollectionTime", "CollectionCount"] + tag_keys = ["name"] + + [[inputs.jolokia2_agent.metric]] + name = "java_last_garbage_collection" + mbean = "java.lang:name=G1 Young Generation,type=GarbageCollector" + paths = ["LastGcInfo/duration", "LastGcInfo/GcThreadCount", "LastGcInfo/memoryUsageAfterGc"] + + [[inputs.jolokia2_agent.metric]] + name = "java_threading" + mbean = "java.lang:type=Threading" + paths = ["TotalStartedThreadCount", "ThreadCount", "DaemonThreadCount", "PeakThreadCount"] + + [[inputs.jolokia2_agent.metric]] + name = "java_class_loading" + mbean = "java.lang:type=ClassLoading" + paths = ["LoadedClassCount", "UnloadedClassCount", "TotalLoadedClassCount"] + + [[inputs.jolokia2_agent.metric]] + name = "java_memory_pool" + mbean = "java.lang:name=*,type=MemoryPool" + paths = ["Usage", "PeakUsage", "CollectionUsage"] + tag_keys = ["name"] + diff --git a/stable/metabase/templates/deployment.yaml b/stable/metabase/templates/deployment.yaml new file mode 100644 index 00000000..d2dfe541 --- /dev/null +++ b/stable/metabase/templates/deployment.yaml @@ -0,0 +1,182 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "metabase.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} + run: {{ .Release.Name }} +spec: + replicas: {{ .Values.replicaCount }} + selector: + matchLabels: + run: {{ .Release.Name }} + template: + metadata: + annotations: + checksum/config: {{ include (print $.Template.BasePath "/config.yaml") . | sha256sum }} + {{- if .Values.podAnnotations }} +{{ toYaml .Values.podAnnotations | trim | indent 8 }} + {{- end }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 8 }} + {{- end }} + run: {{ .Release.Name }} + spec: + initContainers: + {{- range $initContainer := .Values.initContainers }} + - {{- toYaml $initContainer | nindent 10 }} + {{- end }} + containers: + - name: metabase + image: "{{ .Values.metabase.image.repository }}:{{ .Values.metabase.image.tag }}" + imagePullPolicy: {{ .Values.metabase.image.pullPolicy }} + ports: + - containerPort: {{ .Values.service.internalPort }} + protocol: TCP + livenessProbe: + httpGet: + path: {{ .Values.metabase.livenessProbe.path }} + port: {{ .Values.service.internalPort }} + initialDelaySeconds: {{ .Values.metabase.livenessProbe.initialDelaySeconds }} + timeoutSeconds: {{ .Values.metabase.livenessProbe.timeoutSeconds }} + failureThreshold: {{ .Values.metabase.livenessProbe.failureThreshold }} + readinessProbe: + httpGet: + path: {{ .Values.metabase.readinessProbe.path }} + port: {{ .Values.service.internalPort }} + initialDelaySeconds: {{ .Values.metabase.readinessProbe.initialDelaySeconds }} + timeoutSeconds: {{ .Values.metabase.readinessProbe.timeoutSeconds }} + periodSeconds: {{ .Values.metabase.readinessProbe.periodSeconds }} + resources: + {{- toYaml .Values.metabase.resources | nindent 12 }} + env: + - name: MB_JETTY_HOST + value: {{ .Values.metabase.listen.host | quote }} + - name: MB_JETTY_PORT + value: {{ .Values.metabase.listen.port | quote }} + {{- if .Values.metabase.ssl.enabled }} + - name: MB_JETTY_SSL + value: true + - name: MB_JETTY_SSL_Port + value: {{ .Values.metabase.ssl.port | quote }} + - name: MB_JETTY_SSL_Keystore + valueFrom: + secretKeyRef: + name: {{ template "metabase.fullname" . }}-ssl + key: keystore + - name: MB_JETTY_SSL_Keystore_Password + valueFrom: + secretKeyRef: + name: {{ template "metabase.fullname" . }}-ssl + key: password + {{- end }} + - name: MB_DB_TYPE + value: {{ .Values.metabase.database.type | quote }} + {{- if ne (.Values.metabase.database.type | lower) "h2" }} + {{- if or .Values.metabase.database.existingSecretConnectionURIKey .Values.metabase.database.connectionURI }} + - name: MB_DB_CONNECTION_URI + valueFrom: + secretKeyRef: + name: {{ or .Values.metabase.database.existingSecret (printf "%s-database" (include "metabase.fullname" .)) }} + key: {{ or .Values.metabase.database.existingSecretConnectionURIKey "connectionURI" }} + {{- else }} + - name: MB_DB_HOST + value: {{ .Values.metabase.database.host | quote }} + - name: MB_DB_PORT + value: {{ .Values.metabase.database.port | quote }} + - name: MB_DB_DBNAME + value: {{ .Values.metabase.database.dbname | quote }} + - name: MB_DB_USER + valueFrom: + secretKeyRef: + name: {{ or .Values.metabase.database.existingSecret (printf "%s-database" (include "metabase.fullname" .)) }} + key: {{ or .Values.metabase.database.existingSecretUsernameKey "username" }} + - name: MB_DB_PASS + valueFrom: + secretKeyRef: + name: {{ or .Values.metabase.database.existingSecret (printf "%s-database" (include "metabase.fullname" .)) }} + key: {{ or .Values.metabase.database.existingSecretPasswordKey "password" }} + {{- end }} + {{- end }} + - name: MB_PASSWORD_COMPLEXITY + value: {{ .Values.metabase.password.complexity }} + - name: MB_PASSWORD_LENGTH + value: {{ .Values.metabase.password.length | quote }} + - name: JAVA_TIMEZONE + value: {{ .Values.metabase.timeZone }} + {{- if .Values.javaOpts }} + - name: JAVA_OPTS + value: {{ .Values.metabase.javaOpts | quote }} + {{- else }} + {{- if .Values.metabase.log4jProperties }} + - name: JAVA_OPTS + value: "-Dlog4j.configuration=file:/tmp/conf/log4j.properties" + {{- end }} + {{- end }} + {{- if .Values.metabase.pluginsDirectory }} + - name: MB_PLUGINS_DIR + value: {{ .Values.metabase.pluginsDirectory | quote }} + {{- end }} + - name: MB_EMOJI_IN_LOGS + value: {{ .Values.metabase.emojiLogging | quote }} + {{- if .Values.metabase.siteUrl }} + - name: MB_SITE_URL + value: {{ .Values.metabase.siteUrl | quote }} + {{- end }} + {{- if .Values.metabase.secretConfig }} + - name: NEW_RELIC_APP_NAME + valueFrom: + secretKeyRef: + name: {{ (printf "%s-secret" (include "metabase.fullname" .)) }} + key: "newRelicAppName" + - name: NEW_RELIC_LICENSE_KEY + valueFrom: + secretKeyRef: + name: {{ (printf "%s-secret" (include "metabase.fullname" .)) }} + key: "newRelicLicenseKey" + {{- end }} + volumeMounts: + {{- toYaml .Values.metabase.volumeMounts | nindent 12 }} + {{- if .Values.log4jProperties }} + - name: config + mountPath: /tmp/conf/ + {{- end}} + - name: telegraf + image: "{{ .Values.telegraf.image.repository }}:{{ .Values.telegraf.image.tag }}" + imagePullPolicy: {{ .Values.telegraf.image.pullPolicy }} + resources: + {{- toYaml .Values.telegraf.resources | nindent 12 }} + volumeMounts: + - name: telegraf-volume + mountPath: /etc/telegraf + {{- with .Values.nodeSelector }} + nodeSelector: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: +{{ toYaml . | indent 8 }} + {{- end }} + volumes: + {{- range $volume := .Values.volumes }} + - {{- toYaml $volume | nindent 10 }} + {{- end }} + - name: telegraf-volume + configMap: + name: {{ template "metabase.fullname" . }}-telegraf-config + {{- if .Values.log4jProperties}} + - name: config + configMap: + name: {{ template "metabase.fullname" . }}-config + items: + - key: log4j.properties + path: log4j.properties + {{- end }} \ No newline at end of file diff --git a/stable/metabase/templates/ingress.yaml b/stable/metabase/templates/ingress.yaml new file mode 100644 index 00000000..72399864 --- /dev/null +++ b/stable/metabase/templates/ingress.yaml @@ -0,0 +1,38 @@ +{{- if .Values.ingress.enabled -}} +{{- $serviceName := include "metabase.fullname" . -}} +{{- $servicePort := .Values.service.externalPort -}} +{{- $ingressPath := .Values.ingress.path -}} +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ template "metabase.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- range $key, $value := .Values.ingress.labels }} +{{ $key }}: {{ $value | quote | indent 4}} + {{- end }} + run: {{ .Release.Name }} + annotations: + {{- range $key, $value := .Values.ingress.annotations }} + {{ $key }}: {{ $value | quote }} + {{- end }} +spec: + ingressClassName: {{ .Values.ingress.ingressClassName }} + rules: + {{- range $host := .Values.ingress.hosts }} + - host: {{ $host }} + http: + paths: + - path: {{ $ingressPath }} + pathType: "Prefix" + backend: + service: + name: {{ $serviceName }} + port: + number: {{ $servicePort }} + {{- end -}} + {{- if .Values.ingress.tls }} + tls: +{{ toYaml .Values.ingress.tls | indent 4 }} + {{- end -}} +{{- end -}} diff --git a/stable/metabase/templates/secret.yaml b/stable/metabase/templates/secret.yaml new file mode 100644 index 00000000..0c63c5ed --- /dev/null +++ b/stable/metabase/templates/secret.yaml @@ -0,0 +1,60 @@ +{{- if .Values.metabase.secretConfig -}} +apiVersion: v1 +kind: Secret +metadata: + name: {{ template "metabase.fullname" . }}-secret + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} +type: Opaque +data: + {{- if .Values.metabase.secretConfig.newRelicAppName }} + newRelicAppName: {{ .Values.metabase.secretConfig.newRelicAppName | b64enc | quote }} + {{- end }} + {{- if .Values.metabase.secretConfig.newRelicLicenseKey }} + newRelicLicenseKey: {{ .Values.metabase.secretConfig.newRelicLicenseKey | b64enc | quote }} + {{- end }} +{{- end }} + +--- +{{- if and (ne (.Values.metabase.database.type | lower) "h2") (not .Values.metabase.database.existingSecret) }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ template "metabase.fullname" . }}-database + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} +type: Opaque +data: + {{- if .Values.metabase.database.encryptionKey }} + encryptionKey: {{ .Values.metabase.database.encryptionKey | b64enc | quote }} + {{- end }} + {{- if .Values.metabase.database.connectionURI }} + connectionURI: {{ .Values.metabase.database.connectionURI | b64enc | quote }} + {{- else }} + username: {{ .Values.metabase.database.username | b64enc | quote }} + password: {{ .Values.metabase.database.password | b64enc | quote }} + {{- end }} +{{- end }} + +--- +{{- if .Values.metabase.ssl.enabled }} +apiVersion: v1 +kind: Secret +metadata: + name: {{ template "metabase.fullname" . }}-ssl + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} +type: Opaque +data: + keystore: {{ .Values.metabase.ssl.keystore | b64enc | quote }} + password: {{ .Values.metabase.ssl.keyStorePassword | b64enc | quote }} +{{- end }} diff --git a/stable/metabase/templates/service.yaml b/stable/metabase/templates/service.yaml new file mode 100644 index 00000000..4e19fda0 --- /dev/null +++ b/stable/metabase/templates/service.yaml @@ -0,0 +1,30 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "metabase.fullname" . }} + namespace: {{ .Release.Namespace }} + labels: + {{- if .Values.labels }} +{{ toYaml .Values.labels | trim | indent 4 }} + {{- end }} + {{- if .Values.service.annotations }} + run: {{ .Release.Name }} + annotations: +{{ toYaml .Values.service.annotations | indent 4 }} + {{- end }} +spec: + type: {{ .Values.service.type }} +{{- if .Values.service.loadBalancerSourceRanges}} + loadBalancerSourceRanges: +{{toYaml .Values.service.loadBalancerSourceRanges | indent 4 }} +{{- end}} + ports: + - port: {{ .Values.service.externalPort }} + targetPort: {{ .Values.service.internalPort }} +{{- if .Values.service.nodePort }} + nodePort: {{ .Values.service.nodePort }} +{{- end}} + protocol: TCP + name: {{ template "metabase.name" . }} + selector: + run: {{ .Release.Name }} diff --git a/stable/metabase/values.yaml b/stable/metabase/values.yaml new file mode 100644 index 00000000..e2204030 --- /dev/null +++ b/stable/metabase/values.yaml @@ -0,0 +1,167 @@ +replicaCount: 1 +podAnnotations: {} +labels: {} +## String to fully override metabase.fullname template +## +# fullnameOverride: +initContainers: {} +volumes: {} + +metabase: + image: + repository: metabase/metabase + tag: v0.50.10 + pullPolicy: IfNotPresent + + # Config Jetty web server + listen: + host: "0.0.0.0" + port: 3000 + ssl: + # If you have an ssl certificate and would prefer to have Metabase run over HTTPS + enabled: false + # port: 8443 + # keyStore: |- + # << JKS KEY STORE >> + # keyStorePassword: storepass + jetty: + # maxThreads: 254 + # minThreads: 8 + # maxQueued: -1 + # maxIdleTime: 60000 + + resources: {} + # We usually recommend not to specify default resources and to leave this as a conscious + # choice for the user. This also increases chances charts run on environments with little + # resources, such as Minikube. If you do want to specify resources, uncomment the following + # lines, adjust them as necessary, and remove the curly braces after 'resources:'. + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + + # volumeMounts: + # - mountPath: /data/config.yaml + # name: config + # subPath: config.yaml + + # Backend database + database: + # Database type (h2 / mysql / postgres), default: postgres + type: postgres + # encryptionKey: << YOUR ENCRYPTION KEY >> + ## Only need when you use mysql / postgres + host: localhost + port: 5432 + dbname: db_name + username: db_username + password: db_password + ## Alternatively, use a connection URI for full configurability. Example for SSL enabled Postgres. + # connectionURI: postgres://user:password@host:port/database?ssl=true&sslmode=require&sslfactory=org.postgresql.ssl.NonValidatingFactory" + ## If a secret with the database credentials already exists, use the following values: + # existingSecret: + # existingSecretUsernameKey: + # existingSecretPasswordKey: + # existingSecretConnectionURIKey: + + password: + # Changing Metabase password complexity: + # weak: no character constraints + # normal: at least 1 digit (default) + # strong: minimum 8 characters w/ 2 lowercase, 2 uppercase, 1 digit, and 1 special character + complexity: weak + length: 6 + + # secretConfig: + # newRelicAppName: metabase + # newRelicLicenseKey: xxx-xxx-xxx + + timeZone: UTC + emojiLogging: true + # javaOpts: + # pluginsDirectory: + # siteUrl: + + livenessProbe: + path: /api/health + initialDelaySeconds: 120 + timeoutSeconds: 30 + failureThreshold: 6 + + readinessProbe: + path: /api/health + initialDelaySeconds: 30 + timeoutSeconds: 3 + periodSeconds: 5 + +telegraf: + image: + repository: telegraf + tag: 1.19.3-alpine + pullPolicy: IfNotPresent + resources: {} + # limits: + # cpu: 10m + # memory: 30Mi + # requests: + # cpu: 10m + # memory: 30Mi + # influx: + # address: localhost:8086 + # databaseName: influx-database + # prometheusRemoteWrite: + # url: http://remote/api/push + +service: + type: ClusterIP + # loadBalancerSourceRanges: + externalPort: 80 + internalPort: 3000 + # Used to fix NodePort when service.type: NodePort. + nodePort: + annotations: {} + # Used to add custom annotations to the Service. + # service.beta.kubernetes.io/aws-load-balancer-internal: "0.0.0.0/0" + +ingress: + enabled: true + ingressClassName: contour + # Used to create Ingress record (should used with service.type: ClusterIP). + hosts: + # - metabase.domain.com + # The ingress path. Useful to host metabase on a subpath, such as `/metabase`. + path: / + labels: + # Used to add custom labels to the Ingress + # Useful if for example you have multiple Ingress controllers and want your Ingress controllers to bind to specific Ingresses + # traffic: internal + annotations: {} + # kubernetes.io/ingress.class: nginx + # kubernetes.io/tls-acme: "true" + tls: + # Secrets must be manually created in the namespace. + # - secretName: metabase-tls + # hosts: + # - metabase.domain.com + +# A custom log4j.properties file can be provided using a multiline YAML string. +# See https://github.com/metabase/metabase/blob/master/resources/log4j.properties +# +# log4jProperties: + +## Node labels for pod assignment +## ref: https://kubernetes.io/docs/user-guide/node-selection/ +# +nodeSelector: {} + +## Tolerations for pod assignment +## ref: https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/ +## +tolerations: [] + +## Affinity for pod assignment +## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#affinity-and-anti-affinity +## +affinity: {} diff --git a/stable/meteor/Chart.yaml b/stable/meteor/Chart.yaml index d7adb410..d20f45c9 100644 --- a/stable/meteor/Chart.yaml +++ b/stable/meteor/Chart.yaml @@ -1,5 +1,5 @@ apiVersion: v1 -version: 0.1.6 -description: A Helm chart for Meteor (github.com/odpf/meteor) +version: 0.3.7 +description: A Helm chart for Meteor (github.com/goto/meteor) name: meteor -appVersion: "v0.5.1" +appVersion: "v0.8.9" diff --git a/stable/meteor/README.md b/stable/meteor/README.md index db04897a..13a2ec1c 100644 --- a/stable/meteor/README.md +++ b/stable/meteor/README.md @@ -1,6 +1,6 @@ # Meteor -A [helm](https://helm.sh/) chart for [Meteor](https://github.com/odpf/meteor) +A [helm](https://helm.sh/) chart for [Meteor](https://github.com/goto/meteor) ## TL;DR; @@ -10,7 +10,7 @@ $ helm install [my-release] . ## Introduction -This chart can be used for deploying [Meteor](https://github.com/odpf/meteor) on kubenetes using helm +This chart can be used for deploying [Meteor](https://github.com/goto/meteor) on kubenetes using helm ## Installing the Chart @@ -47,11 +47,17 @@ The following table lists the configurable parameters of the Meteor chart and th | recipes | object {\[filename\]: \[content\]} | [example](#sample-recipes-usage) | | | namespace | string | `-` | to override release namespace | | image.pullPolicy | string | `"IfNotPresent"` | | -| image.repository | string | `"odpf/meteor"` | Image to use for deploying siren | -| image.tag | string | `"0.1.0"` | | +| image.repository | string | `"gotocompany/meteor"` | Image to use for deploying meteor | +| image.tag | string | `"0.8.0"` | | | labels | object | `{}` | | | config | object | `{}` | | | secretConfig | object | `{}` | secrets | +| telegraf.enabled | bool | `false` | | +| telegraf.command | list | `["telegraf"]` | | +| telegraf.args | list | `[]` | | +| telegraf.containerPort | int | `8125` | | +| telegraf.protocol | string | `UDP` | | +| telegraf.config | string | `""` | telegraf config file content | ### Using recipes values ``` diff --git a/stable/meteor/templates/cronjob.yaml b/stable/meteor/templates/cronjob.yaml index 76e3b053..65ee4f28 100644 --- a/stable/meteor/templates/cronjob.yaml +++ b/stable/meteor/templates/cronjob.yaml @@ -1,4 +1,4 @@ -apiVersion: batch/v1beta1 +apiVersion: batch/v1 kind: CronJob metadata: name: "{{ include "meteor.name" . }}" @@ -13,21 +13,49 @@ spec: jobTemplate: spec: backoffLimit: 4 + activeDeadlineSeconds: {{ .Values.jobDeadlineSeconds }} template: metadata: + {{- with .Values.annotations }} + annotations: + {{- toYaml . | nindent 12 }} + {{- end }} labels: {{- if .Values.labels }} {{ toYaml .Values.labels | indent 12 }} {{- end }} spec: + {{- if .Values.serviceAccount.name }} + serviceAccountName: {{ .Values.serviceAccount.name }} + {{- end }} containers: - name: "{{ include "meteor.name" . }}" image: "{{ required `image.repository is required` .Values.image.repository }}:{{ .Values.image.tag }}" imagePullPolicy: {{ .Values.image.pullPolicy }} command: - - meteor - - run - - /opt/recipes + - /bin/sh + - -c + - | + set -o pipefail + meteor run /opt/recipes | tee -a output.txt + METEOR_EXIT_CODE=$? + {{- if .Values.postCmd.enabled }} + NUM_FAILING=$(cat output.txt | grep -o '^[0-9]* failing' | awk '{print $1}') + # if the output does not contains number of failing, then it is failed + if [ -z $NUM_FAILING ]; then + NUM_FAILING=-1 + fi + # run post cmd only if exit code is success and no any failing occurred in meteor job + if [ $METEOR_EXIT_CODE -eq 0 ] && [ $NUM_FAILING -eq 0 ]; then + {{- .Values.postCmd.cmd | nindent 18 }} + fi + {{- end }} + {{- if .Values.telegraf.enabled }} + pkill -SIGTERM telegraf + {{- end }} + {{- if .Values.otelcollector.enabled }} + pkill -SIGTERM otelcol-contrib + {{- end }} volumeMounts: - name: "{{ include "meteor.name" . }}-volume" mountPath: /opt/recipes @@ -35,6 +63,10 @@ spec: - name: {{ $v }} mountPath: {{ printf "%s/%s" "/etc/secret" $v }} {{- end }} + {{- if .Values.serviceAccount.tokenProjection.enabled }} + - name: {{ .Values.serviceAccount.name }} + mountPath: {{ .Values.serviceAccount.tokenProjection.mountPath }} + {{- end}} envFrom: - configMapRef: name: "{{ include "meteor.name" . }}-variables-configmap" @@ -42,6 +74,83 @@ spec: - secretRef: name: "{{ include "meteor.name" . }}-secret" {{- end }} + resources: + {{- toYaml .Values.resources | nindent 14 }} + {{- if .Values.otelcollector.enabled }} + - name: otel-collector-sidecar + image: {{ .Values.otelcollector.image }} + imagePullPolicy: IfNotPresent + args: + - "--config=/etc/otelcollector/config.yaml" + ports: + - containerPort: 1888 + name: pprof + protocol: TCP + - containerPort: 8888 + name: prometheusself + protocol: TCP + - containerPort: 8889 + name: prometheus + protocol: TCP + - containerPort: 13133 + name: healthcheck + protocol: TCP + - containerPort: 4317 + name: otlpgrpcrecv + protocol: TCP + - containerPort: 4318 + name: otlphttprecv + protocol: TCP + - containerPort: 55679 + name: zpages + protocol: TCP + env: + - name: K8S_NODE_NAME + valueFrom: + fieldRef: + fieldPath: spec.nodeName + - name: K8S_POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: K8S_POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + - name: K8S_POD_IP + valueFrom: + fieldRef: + fieldPath: status.podIP + - name: OTEL_RESOURCE_ATTRIBUTES + value: | + k8s.deployment.name={{ include "meteor.name" . }},k8s.namespace.name=$(K8S_POD_NAMESPACE),k8s.node.name=$(K8S_NODE_NAME),k8s.pod.name=$(K8S_POD_NAME),k8s.pod.ip=$(K8S_POD_IP) + volumeMounts: + - name: otelcollector-conf + mountPath: /etc/otelcollector/ + {{- end }} + {{- if .Values.telegraf.enabled }} + - name: telegraf-sidecar + image: {{ .Values.telegraf.image }} + args: + {{- range .Values.telegraf.args }} + - {{ . }} + {{- end }} + command: + {{- range .Values.telegraf.command }} + - {{ . }} + {{- end }} + imagePullPolicy: IfNotPresent + ports: + - containerPort: {{ .Values.telegraf.containerPort }} + name: http + protocol: {{ .Values.telegraf.protocol }} + volumeMounts: + - name: telegraf-conf + mountPath: /etc/telegraf/ + {{- end }} + {{- if or .Values.telegraf.enabled .Values.otelcollector.enabled }} + shareProcessNamespace: true + {{- end}} restartPolicy: Never volumes: - name: "{{ include "meteor.name" . }}-volume" @@ -59,3 +168,22 @@ spec: optional: true secretName: {{ $v }} {{- end }} + {{- if .Values.telegraf.enabled }} + - name: telegraf-conf + configMap: + name: {{ include "meteor.name" . }}-telegraf + {{- end }} + {{- if .Values.otelcollector.enabled }} + - name: otelcollector-conf + configMap: + name: {{ include "meteor.name" . }}-otelcollector + {{- end }} + {{- if .Values.serviceAccount.tokenProjection.enabled }} + - name: {{ .Values.serviceAccount.name }} + projected: + sources: + - serviceAccountToken: + path: {{ .Values.serviceAccount.tokenProjection.subPath }} + expirationSeconds: {{ .Values.serviceAccount.tokenProjection.expirationSeconds }} + audience: {{ .Values.serviceAccount.tokenProjection.audience }} + {{- end}} diff --git a/stable/meteor/templates/otelcollector-configmap.yaml b/stable/meteor/templates/otelcollector-configmap.yaml new file mode 100644 index 00000000..8cccea25 --- /dev/null +++ b/stable/meteor/templates/otelcollector-configmap.yaml @@ -0,0 +1,13 @@ +{{- if .Values.otelcollector.enabled -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "meteor.name" . }}-otelcollector + labels: +{{- if .Values.labels }} +{{ toYaml .Values.labels | indent 4 }} +{{- end }} +data: + config.yaml: |- +{{ .Values.otelcollector.config | indent 4 }} +{{- end }} diff --git a/stable/meteor/templates/telegraf-configmap.yaml b/stable/meteor/templates/telegraf-configmap.yaml new file mode 100644 index 00000000..161e272e --- /dev/null +++ b/stable/meteor/templates/telegraf-configmap.yaml @@ -0,0 +1,13 @@ +{{- if .Values.telegraf.enabled -}} +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ include "meteor.name" . }}-telegraf + labels: +{{- if .Values.labels }} +{{ toYaml .Values.labels | indent 4 }} +{{- end }} +data: + telegraf.conf: |- +{{ .Values.telegraf.config | indent 4 }} +{{- end }} diff --git a/stable/meteor/values.yaml b/stable/meteor/values.yaml index f8863a86..b05b54c1 100644 --- a/stable/meteor/values.yaml +++ b/stable/meteor/values.yaml @@ -2,8 +2,8 @@ schedule: "0 1 * * *" concurrencyPolicy: "Forbid" image: # [required] repository where meteor Docker image is available - repository: odpf/meteor - tag: "0.5.1" + repository: gotocompany/meteor + tag: "0.8.0" pullPolicy: IfNotPresent labels: {"application":"meteor"} @@ -12,6 +12,24 @@ secretConfig: {} # pass in secret names to be mounted to cronjob ssl_secrets: [] +annotations: {} + +jobDeadlineSeconds: 14400 + +resources: + {} + # -- We usually recommend not to specify default resources and to leave this as a conscious + # -- choice for the user. This also increases chances charts run on environments with little + # -- resources, such as Minikube. If you do want to specify resources, uncomment the following + # -- lines, adjust them as necessary, and remove the curly braces after 'resources:'. + # -- limits: + # -- cpu: 100m + # -- memory: 128Mi + # -- requests: + # -- cpu: 100m + # -- memory: 128Mi + + # sample recipe usage # recipes: # sample-recipe.yaml: |- @@ -22,3 +40,44 @@ ssl_secrets: [] # foo: "bar" # sinks: # - name: console +postCmd: + enabled: false + cmd: "" + +telegraf: + enabled: false + image: telegraf:1.19.3-alpine + config: ~ # telegraf config + containerPort: 8125 + protocol: UDP + args: + command: + - telegraf + +otelcollector: + enabled: false + image: otel/opentelemetry-collector-contrib:0.79.0 + config: | + receivers: + otlp: + protocols: + grpc: + http: + processors: + exporters: + logging: + service: + pipelines: + traces: + receivers: [otlp] + processors: [] + exporters: [logging] + +serviceAccount: + name: "" + tokenProjection: + enabled: false + mountPath: "/var/run/secrets" + subPath: "token" + expirationSeconds: 600 + audience: "" diff --git a/stable/optimus/Chart.yaml b/stable/optimus/Chart.yaml index 703e43c8..4ab37e67 100644 --- a/stable/optimus/Chart.yaml +++ b/stable/optimus/Chart.yaml @@ -9,10 +9,10 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.0.3 +version: 0.0.10 # This is the version number of the application being deployed. This version number should be # incremented each time you make changes to the application. Versions are not expected to # follow Semantic Versioning. They should reflect the version the application is using. # It is recommended to use it with quotes. -appVersion: "0.3.0" +appVersion: "0.11.1" diff --git a/stable/optimus/templates/_helpers.tpl b/stable/optimus/templates/_helpers.tpl index 70c801d6..e42cfb65 100644 --- a/stable/optimus/templates/_helpers.tpl +++ b/stable/optimus/templates/_helpers.tpl @@ -40,6 +40,9 @@ helm.sh/chart: {{ include "app.chart" . }} app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} {{- end }} app.kubernetes.io/managed-by: {{ .Release.Service }} +{{- range $key, $value := .Values.app.labels }} +{{ $key }}: {{ $value }} +{{- end }} {{- end }} {{/* diff --git a/stable/optimus/templates/configmap.yaml b/stable/optimus/templates/configmap.yaml index cf1caf19..a770b150 100644 --- a/stable/optimus/templates/configmap.yaml +++ b/stable/optimus/templates/configmap.yaml @@ -19,3 +19,15 @@ metadata: "helm.sh/hook-weight": "-5" data: config.yaml: {{ toYaml .Values.configYaml | indent 4 }} +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: {{ template "app.fullname" . }}-plugin-yaml + annotations: + "helm.sh/hook": pre-install,pre-upgrade + "helm.sh/hook-weight": "-5" +data: + {{- range $k, $v := .Values.pluginYaml }} + {{ $k }}: {{ toYaml $v | indent 4 }} + {{- end }} diff --git a/stable/optimus/templates/data-cleanup-cronjob.yaml b/stable/optimus/templates/data-cleanup-cronjob.yaml new file mode 100644 index 00000000..8c32304a --- /dev/null +++ b/stable/optimus/templates/data-cleanup-cronjob.yaml @@ -0,0 +1,30 @@ +{{- if .Values.cleanupConfig.enabled -}} +apiVersion: batch/v1 +kind: CronJob +metadata: + name: {{ include "app.fullname" . }}-cleanup + labels: + {{- include "app.labels" . | nindent 4 }} +spec: + schedule: {{ .Values.cleanupConfig.schedule }} + concurrencyPolicy: {{ .Values.cleanupConfig.concurrencyPolicy }} + successfulJobsHistoryLimit: {{ .Values.cleanupConfig.successJobHistoryLimit }} + jobTemplate: + spec: + template: + metadata: + labels: + {{- include "app.labels" . | nindent 12 }} + spec: + containers: + - name: {{ include "app.fullname" . }}-cleanup + image: {{ .Values.cleanupConfig.image }} + envFrom: + - configMapRef: + name: {{ template "app.fullname" . }} + env: + - name: QUERIES + value: "{{- range .Values.cleanupConfig.queries -}} {{.}} {{- end }}" + command: ["sh", "-c", "set -x; echo $QUERIES; psql $OPTIMUS_SERVE_DB_DSN -c \"$QUERIES\";"] + restartPolicy: {{ .Values.cleanupConfig.restartPolicy }} +{{- end }} \ No newline at end of file diff --git a/stable/optimus/templates/deployment.yaml b/stable/optimus/templates/deployment.yaml index b5501a8c..00ecd908 100644 --- a/stable/optimus/templates/deployment.yaml +++ b/stable/optimus/templates/deployment.yaml @@ -25,6 +25,7 @@ spec: {{- end }} labels: {{- include "app.selectorLabels" . | nindent 8 }} + {{- include "app.labels" . | nindent 8 }} spec: {{- with .Values.imagePullSecrets }} imagePullSecrets: @@ -58,6 +59,9 @@ spec: - mountPath: /etc/optimus name: configuration-volume readOnly: true + - mountPath: /etc/optimus-plugins + name: plugins-volume + readOnly: true {{- range $volumeMount := .Values.container.volumeMounts }} - {{- toYaml $volumeMount | nindent 16 }} {{- end }} @@ -110,6 +114,9 @@ spec: - name: configuration-volume configMap: name: {{ template "app.fullname" . }}-yaml + - name: plugins-volume + configMap: + name: {{ template "app.fullname" . }}-plugin-yaml {{- range $volume := .Values.volumes }} - {{- toYaml $volume | nindent 12 }} {{- end }} diff --git a/stable/optimus/templates/ingress.yaml b/stable/optimus/templates/ingress.yaml index a34de5fe..dbdc1338 100644 --- a/stable/optimus/templates/ingress.yaml +++ b/stable/optimus/templates/ingress.yaml @@ -1,28 +1,18 @@ {{- if .Values.ingress.enabled -}} - {{- $fullName := include "app.fullname" . -}} - {{- if and .Values.ingress.className (not (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion)) }} - {{- if not (hasKey .Values.ingress.annotations "kubernetes.io/ingress.class") }} - {{- $_ := set .Values.ingress.annotations "kubernetes.io/ingress.class" .Values.ingress.className}} - {{- end }} - {{- end }} - {{- if semverCompare ">=1.19-0" .Capabilities.KubeVersion.GitVersion -}} +{{- $fullName := include "app.fullname" . -}} +{{- $labels := include "app.labels" . -}} apiVersion: networking.k8s.io/v1 - {{- else if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}} -apiVersion: networking.k8s.io/v1beta1 - {{- else -}} -apiVersion: extensions/v1beta1 - {{- end }} kind: Ingress metadata: name: {{ $fullName }} labels: - {{- include "app.labels" . | nindent 4 }} + {{- $labels | nindent 4 }} {{- with .Values.ingress.annotations }} annotations: {{- toYaml . | nindent 4 }} {{- end }} spec: - {{- if and .Values.ingress.className (semverCompare ">=1.18-0" .Capabilities.KubeVersion.GitVersion) }} + {{- if .Values.ingress.className }} ingressClassName: {{ .Values.ingress.className }} {{- end }} {{- if .Values.ingress.tls }} @@ -43,19 +33,56 @@ spec: paths: {{- range .paths }} - path: {{ .path }} - {{- if and .pathType (semverCompare ">=1.18-0" $.Capabilities.KubeVersion.GitVersion) }} pathType: {{ .pathType }} - {{- end }} backend: - {{- if semverCompare ">=1.19-0" $.Capabilities.KubeVersion.GitVersion }} service: name: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} port: number: {{ .backend.service.port.number }} - {{- else }} - serviceName: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} - servicePort: {{ .backend.service.port.number }} - {{- end }} + {{- end }} + {{- end }} +{{- if .Values.ingress.additionalIngress }} + {{- range .Values.ingress.additionalIngress }} +--- +apiVersion: networking.k8s.io/v1 +kind: Ingress +metadata: + name: {{ printf "%s-%s" $fullName .name }} + labels: + {{- $labels | nindent 4 }} + {{- with .annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + {{- if .className }} + ingressClassName: {{ .className }} + {{- end }} + {{- if .tls }} + tls: + {{- range .tls }} + - hosts: + {{- range .hosts }} + - {{ . | quote }} + {{- end }} + secretName: {{ .secretName }} + {{- end }} + {{- end }} + rules: + {{- range .hosts }} + - host: {{ .host | quote }} + http: + paths: + {{- range .paths }} + - path: {{ .path }} + pathType: {{ .pathType }} + backend: + service: + name: {{ ternary $fullName .backend.service.name (empty .backend.service.name) }} + port: + number: {{ .backend.service.port.number }} {{- end }} {{- end }} {{- end }} +{{- end }} +{{- end }} \ No newline at end of file diff --git a/stable/optimus/values.yaml b/stable/optimus/values.yaml index c4f3ba3e..71b6396b 100644 --- a/stable/optimus/values.yaml +++ b/stable/optimus/values.yaml @@ -16,8 +16,8 @@ deployment: type: RollingUpdate image: - repository: odpf/optimus - tag: "0.2.5" + repository: gotocompany/optimus + tag: "0.11.1" pullPolicy: Always container: @@ -25,9 +25,12 @@ container: securityContext: runAsUser: 1000 ports: - - name: optimus + - name: http containerPort: 9100 protocol: TCP + - name: grpc + containerPort: 9101 + protocol: TCP command: - optimus args: @@ -37,14 +40,14 @@ container: livenessProbe: httpGet: path: /ping - port: optimus - initialDelaySeconds: 60 + port: http + initialDelaySeconds: 30 timeoutSeconds: 1 readinessProbe: httpGet: path: /ping - port: optimus - initialDelaySeconds: 60 + port: http + initialDelaySeconds: 30 timeoutSeconds: 1 # serve.host field in configYaml breaks the live-ness probe @@ -54,7 +57,9 @@ configYaml: |- level: DEBUG serve: port: 9100 + port_grpc: 9101 ingress_host: optimus.optimus.svc.cluster.local + ingress_host_grpc: optimus.grpc.optimus.svc.cluster.local deployer: num_workers: 2 scheduler: @@ -62,24 +67,55 @@ configYaml: |- telemetry: profile_addr: ":9110" jaeger_addr: "http://jaeger-collector:14268/api/traces" + plugin: + artifacts: + - /etc/optimus-plugins config: OPTIMUS_TELEMETRY_JAEGER_ADDR: http://jaeger-collector.optimus:14268/api/traces +pluginYaml: + bq2bq.yaml: |- + name: bq2bq + description: BigQuery to BigQuery transformation task + plugintype: task + pluginversion: 0.4.2 + image: 'docker.io/gotocompany/optimus-task-bq2bq-executor:0.4.2' + entrypoint: + script: "python3 /opt/bumblebee/main.py" + secretConfig: {} +cleanupConfig: + enabled: true + schedule: "0 0 1 * *" # Runs At 00:00 on day-of-month 1 + image: "postgres:15.2-alpine3.17" + restartPolicy: OnFailure + concurrencyPolicy: Forbid + successJobHistoryLimit: 1 + queries: + - "delete from sensor_run where start_time < now() - Interval '5 months';" + - "delete from task_run where start_time < now() - Interval '5 months';" + - "delete from hook_run where start_time < now() - Interval '5 months';" + - "delete from job_run where start_time < now() - Interval '5 months';" + service: type: ClusterIP ports: - port: 80 - targetPort: optimus + targetPort: http protocol: TCP name: http + - port: 8081 + name: grpc + targetPort: grpc + protocol: TCP annotations: - projectcontour.io/upstream-protocol.h2c: 80,http + projectcontour.io/upstream-protocol.h2c: grpc ingress: enabled: false + className: "" annotations: kubernetes.io/ingress.class: contour projectcontour.io/response-timeout: infinity @@ -93,6 +129,20 @@ ingress: name: optimus port: number: 80 + additionalIngress: + - name: grpc + className: "" + annotations: {} + hosts: + - host: optimus-grpc.example.com + paths: + - path: / + pathType: ImplementationSpecific + backend: + service: + name: optimus + port: + number: 9101 telegraf: enabled: false diff --git a/stable/raccoon/README.md b/stable/raccoon/README.md index 8183faa2..0213c9a0 100644 --- a/stable/raccoon/README.md +++ b/stable/raccoon/README.md @@ -1,15 +1,15 @@ # Raccoon -[Raccoon](https://github.com/odpf/raccoon) is high throughput, low-latency service that provides an API to ingest clickstream data from mobile apps, sites and publish it to Kafka. +[Raccoon](https://github.com/goto/raccoon) is high throughput, low-latency service that provides an API to ingest clickstream data from mobile apps, sites and publish it to Kafka. ## TL;DR ``` -$ helm install my-release odpf/raccoon +$ helm install my-release gotocompany/raccoon ``` The service will listen to `:8080` and publish to `host.docker.internal:9093` Kafka by default. ## Introduction -This chart runs [Raccoon](https://github.com/odpf/raccoon) deployment on [Kubernetes](https://kubernetes.io/) using [Helm](https://helm.sh/) package manager. The deployment contains Raccoon service and [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/) as a sidecar. +This chart runs [Raccoon](https://github.com/goto/raccoon) deployment on [Kubernetes](https://kubernetes.io/) using [Helm](https://helm.sh/) package manager. The deployment contains Raccoon service and [Telegraf](https://www.influxdata.com/time-series-platform/telegraf/) as a sidecar. ## Prerequisites - Kubernetes 1.12+ @@ -19,10 +19,10 @@ This chart runs [Raccoon](https://github.com/odpf/raccoon) deployment on [Kubern ### Install released version using Helm repository -**Add the odpf charts repo** +**Add the gotocompany charts repo** ```bash -$ helm repo add odpf https://odpf.github.io/charts/ +$ helm repo add gotocompany https://goto.github.io/charts/ ``` **Install it with Helm 3** @@ -30,7 +30,7 @@ $ helm repo add odpf https://odpf.github.io/charts/ Make sure you are in the correct Kubernetes context. Run the following command to deploy Raccoon. On the following command, you can adjust `PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS` and `SERVER_WEBSOCKET_PORT` as needed. ```bash -$ helm install my-release odpf/raccoon \ +$ helm install my-release gotocompany/raccoon \ --set raccoon.config.PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS=localhost:9092 \ --set raccoon.config.SERVER_WEBSOCKET_PORT=8080 ``` @@ -81,9 +81,9 @@ The following table lists the configurable parameters of Raccoon chart and their | Key | Type | Default | Description | |-----|------|---------|-------------| -| raccoon.config | object | `{"SERVER_WEBSOCKET_PORT": "8080", "SERVER_WEBSOCKET_CONN_UNIQ_ID_HEADER": "x-user-id", "PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS": "host.docker.internal:9094", "METRIC_STATSD_ADDRESS": "host.docker.internal:8125"}` | env variables required by raccoon, [read more](https://odpf.gitbook.io/raccoon/reference/configurations) | +| raccoon.config | object | `{"SERVER_WEBSOCKET_PORT": "8080", "SERVER_WEBSOCKET_CONN_UNIQ_ID_HEADER": "x-user-id", "PUBLISHER_KAFKA_CLIENT_BOOTSTRAP_SERVERS": "host.docker.internal:9094", "METRIC_STATSD_ADDRESS": "host.docker.internal:8125"}` | env variables required by raccoon, [read more](https://goto.gitbook.io/raccoon/reference/configurations) | | raccoon.image.pullPolicy | string | `"IfNotPresent"` | the Kubernetes [imagePullPolicy](https://kubernetes.io/docs/concepts/containers/images/#updating-images) value for raccoon container | -| raccoon.image.repository | string | `"odpf/raccoon"` | docker repository to download raccoon image | +| raccoon.image.repository | string | `"gotocompany/raccoon"` | docker repository to download raccoon image | | raccoon.image.tag | string | `"0.1.0"` | raccoon docker image tag | | raccoon.resources.limits.cpu | string | `"200m"` | raccoon container cpu limit | | raccoon.resources.limits.memory | string | `"512Mi"` | raccoon container memory limit | @@ -121,7 +121,7 @@ Specify each parameter using the `--set key=value[,key=value]` argument to `helm Alternatively, a YAML file that specifies the values for the parameters can be provided while installing the chart. For example, ```bash -$ helm install my-release -f values.yaml odpf/raccoon +$ helm install my-release -f values.yaml gotocompany/raccoon ``` > **Tip**: You can use the default [values.yaml](values.yaml) @@ -131,7 +131,7 @@ The deployment is configurable by [overriding](https://helm.sh/docs/chart_templa ### Enable Telegraf As Sidecar Raccoon uses [statsd](https://www.datadoghq.com/blog/statsd/) to report metrics. Telegraf has statsd input plugin and has Influx output plugin. Once stored in Influx, the metrics can be visualized by using Grafana. More of that you can find it [here](). This chart includes Telegraf as sidecar. By default the sidecar is not enabled. To enable the Telegraf sidecar, you need to set `telegraf.enabled` and `telegraf.config.output.influxdb` to true. -You also need to adjust `telegraf.config.output.influxdb.urls`, `telegraf.config.output.influxdb.database`, and `telegraf.config.output.influxdb.retention_policy`. By default, Raccoon is pointing to `:8086` for [METRIC_STATSD_ADDRESS](https://odpf.gitbook.io/raccoon/reference/configurations#metric_statsd_address). In which, will publish the metrics to the sidecar Telegraf. +You also need to adjust `telegraf.config.output.influxdb.urls`, `telegraf.config.output.influxdb.database`, and `telegraf.config.output.influxdb.retention_policy`. By default, Raccoon is pointing to `:8086` for [METRIC_STATSD_ADDRESS](https://goto.gitbook.io/raccoon/reference/configurations#metric_statsd_address). In which, will publish the metrics to the sidecar Telegraf. Example of Telegraf enabled `values.yaml` ``` telegraf: @@ -150,7 +150,7 @@ telegraf: retention_policy: "autogen" ``` ### Add Raccoon Configurations -Apart from default Raccoon configurations mentioned above, Raccoon also has other application level [configurations](https://odpf.gitbook.io/raccoon/reference/configurations). You can provide the configuration under `raccoon.config.*`. Refer below for example +Apart from default Raccoon configurations mentioned above, Raccoon also has other application level [configurations](https://goto.gitbook.io/raccoon/reference/configurations). You can provide the configuration under `raccoon.config.*`. Refer below for example ``` raccoon: config: diff --git a/stable/raccoon/values.yaml b/stable/raccoon/values.yaml index 45d8b9f8..5147f50c 100644 --- a/stable/raccoon/values.yaml +++ b/stable/raccoon/values.yaml @@ -9,7 +9,7 @@ labels: {"application":"raccoon"} raccoon: image: - repository: odpf/raccoon + repository: gotocompany/raccoon pullPolicy: IfNotPresent tag: 0.1.0 config: diff --git a/stable/shield/Chart.lock b/stable/shield/Chart.lock deleted file mode 100644 index a0f1b428..00000000 --- a/stable/shield/Chart.lock +++ /dev/null @@ -1,9 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.3.1 -- name: app - repository: https://odpf.github.io/charts/ - version: 0.3.1 -digest: sha256:e20a32438123705d7df465ae23b49a281cc2497f8803ca01e47f72435416ae6e -generated: "2022-02-22T16:55:13.364628+05:30" diff --git a/stable/shield/Chart.yaml b/stable/shield/Chart.yaml index 84f475f8..bb769edd 100644 --- a/stable/shield/Chart.yaml +++ b/stable/shield/Chart.yaml @@ -15,16 +15,16 @@ type: application # This is the chart version. This version number should be incremented each time you make changes # to the chart and its templates, including the app version. # Versions are expected to follow Semantic Versioning (https://semver.org/) -version: 0.1.4 +version: 0.2.0 dependencies: -- name: app - version: "0.3.4" - repository: "https://odpf.github.io/charts/" - alias: shield-app - condition: shield-app.enabled -- name: app - version: "0.3.4" - repository: "https://odpf.github.io/charts/" - alias: spicedb - condition: spicedb.enabled + - name: app + version: "0.7.0" + repository: "https://goto.github.io/charts/" + alias: shield-app + condition: shield-app.enabled + - name: app + version: "0.7.0" + repository: "https://goto.github.io/charts/" + alias: spicedb + condition: spicedb.enabled diff --git a/stable/shield/README.MD b/stable/shield/README.MD index f425cb4d..343f8847 100644 --- a/stable/shield/README.MD +++ b/stable/shield/README.MD @@ -8,8 +8,8 @@ A Helm chart for Kubernetes | Repository | Name | Version | | ------------------------------ | --------------- | ------- | -| https://odpf.github.io/charts/ | shield-app(app) | 0.3.0 | -| https://odpf.github.io/charts/ | spicedb(app) | 0.3.0 | +| https://goto.github.io/charts/ | shield-app(app) | 0.3.0 | +| https://goto.github.io/charts/ | spicedb(app) | 0.3.0 | ## Values @@ -30,7 +30,7 @@ A Helm chart for Kubernetes | shield-app.container.readinessProbe.httpGet.port | int | `5556` | | | shield-app.enabled | bool | `true` | | | shield-app.image.pullPolicy | string | `"IfNotPresent"` | | -| shield-app.image.repository | string | `"odpf/shield"` | | +| shield-app.image.repository | string | `"gotocompany/shield"` | | | shield-app.image.tag | string | `"0.2.0-rc"` | | | shield-app.ingress.annotations."kubernetes.io/ingress.class" | string | `"contour"` | | | shield-app.ingress.enabled | bool | `true` | | diff --git a/stable/shield/values.yaml b/stable/shield/values.yaml index 16f5f950..8dc8d303 100644 --- a/stable/shield/values.yaml +++ b/stable/shield/values.yaml @@ -1,12 +1,12 @@ shield-app: enabled: true image: - repository: odpf/shield - pullPolicy: IfNotPresent - tag: 0.2.10 + repository: gotocompany/shield + pullPolicy: Always + tag: 0.7.0 container: args: - - serve + - serve livenessProbe: httpGet: path: /ping @@ -16,12 +16,13 @@ shield-app: path: /ping port: 5556 ports: - - name: tcp - containerPort: 8080 - protocol: TCP - - name: proxy - containerPort: 5556 - protocol: TCP + - name: tcp + containerPort: 8080 + protocol: TCP + - name: proxy + containerPort: 5556 + protocol: TCP + podLabels: {} service: type: ClusterIP ports: @@ -57,28 +58,28 @@ shield-app: SHIELD_SPICEDB_HOST: spicedb.namespace.svc.local SHIELD_SPICEDB_PORT: 50051 secretConfig: - SHIELD_DB_URL: - SHIELD_SPICEDB_PRE_SHARED_KEY: + SHIELD_DB_URL: + SHIELD_SPICEDB_PRE_SHARED_KEY: -spicedb: +spicedb: enabled: true nameOverride: shield-spicedb replicaCount: 2 image: repository: quay.io/authzed/spicedb tag: v1.2.0 - secretConfig: + secretConfig: SPICEDB_GRPC_PRESHARED_KEY: - SPICEDB_DATASTORE_CONN_URI: + SPICEDB_DATASTORE_CONN_URI: config: SPICEDB_LOG_LEVEL: info SPICEDB_DATASTORE_ENGINE: SPICEDB_GRPC_SHUTDOWN_GRACE_PERIOD: "5s" container: ports: - - name: tcp - containerPort: 50051 - protocol: TCP + - name: tcp + containerPort: 50051 + protocol: TCP livenessProbe: httpGet: path: / @@ -88,6 +89,7 @@ spicedb: path: / port: 8080 command: ["spicedb", "serve"] + podLabels: {} ingress: enabled: true annotations: diff --git a/stable/siren/Chart.yaml b/stable/siren/Chart.yaml index 540d3237..3a54b119 100644 --- a/stable/siren/Chart.yaml +++ b/stable/siren/Chart.yaml @@ -1,22 +1,22 @@ apiVersion: v2 name: siren description: Siren Helm chart -version: 0.1.2 -appVersion: v0.5.0 -home: https://github.com/odpf/siren +version: 0.2.0 +appVersion: v0.7.6 +home: https://github.com/goto/siren dependencies: -- name: app - version: "0.4.3" - repository: "https://odpf.github.io/charts/" - alias: app - condition: app.enabled -- name: app - version: "0.4.3" - repository: "https://odpf.github.io/charts/" - alias: notification-worker - condition: notification-worker.enabled -- name: app - version: "0.4.3" - repository: "https://odpf.github.io/charts/" - alias: notification-dlq-worker - condition: notification-dlq-worker.enabled + - name: app + version: "0.6.1" + repository: "https://goto.github.io/charts/" + alias: app + condition: app.enabled + - name: app + version: "0.6.1" + repository: "https://goto.github.io/charts/" + alias: notification-worker + condition: notification-worker.enabled + - name: app + version: "0.6.1" + repository: "https://goto.github.io/charts/" + alias: notification-dlq-worker + condition: notification-dlq-worker.enabled diff --git a/stable/siren/README.md b/stable/siren/README.md index d159cf53..167f04cc 100644 --- a/stable/siren/README.md +++ b/stable/siren/README.md @@ -1,6 +1,6 @@ # Siren -A [helm](https://helm.sh/) chart for [Siren](https://github.com/odpf/siren) +A [helm](https://helm.sh/) chart for [Siren](https://github.com/goto/siren) ## TL;DR; @@ -10,7 +10,7 @@ $ helm install [my-release] . ## Introduction -This chart can be used for deploying [Siren](https://github.com/odpf/siren) on kubenetes using helm +This chart can be used for deploying [Siren](https://github.com/goto/siren) on kubenetes using helm ## Installing the Chart @@ -45,7 +45,7 @@ The following table lists the configurable parameters of the Siren chart and the | affinity | object | `{}` | | | config | object | `{}` | Environement variables needed by siren | | image.pullPolicy | string | `"Always"` | | -| image.repository | string | `"odpf/siren"` | Image to use for deploying siren | +| image.repository | string | `"gotocompany/siren"` | Image to use for deploying siren | | image.tag | string | `"v0.0.1"` | | | ingress.annotations | object | `{"kubernetes.io/ingress.class":"contour"}` | Used to add custom labels to the Ingress Useful if for example you have multiple Ingress controllers and want your Ingress controllers to bind to specific Ingresses traffic: internal | | ingress.enabled | bool | `true` | Kubernetes ingress details | diff --git a/stable/siren/templates/services.yaml b/stable/siren/templates/services.yaml new file mode 100644 index 00000000..b2aeef50 --- /dev/null +++ b/stable/siren/templates/services.yaml @@ -0,0 +1,28 @@ +{{- $dot := . }} +{{- if .Values.app.additionalServices }} +{{- range $i, $service := .Values.app.additionalServices }} +apiVersion: v1 +kind: Service +metadata: + name: {{ include "app.fullname" $dot}}-{{ $service.name }} + labels: + {{- include "app.labels" $dot | nindent 4 }} + {{- with $service.annotations }} + annotations: + {{- toYaml . | nindent 4 }} + {{- end }} +spec: + type: {{ $service.type }} + ports: + {{- range $port := $service.ports }} + - port: {{ $port.port }} + targetPort: {{ $port.targetPort }} + protocol: {{ $port.protocol }} + name: {{ $port.name }} + {{- end }} + selector: + {{- range $k, $v := $service.selectorLabels }} + {{ $k }}: {{ $v }} + {{- end }} +{{- end -}} +{{- end -}} diff --git a/stable/siren/values.yaml b/stable/siren/values.yaml index 4434aa3f..9174ccad 100644 --- a/stable/siren/values.yaml +++ b/stable/siren/values.yaml @@ -1,6 +1,6 @@ app: image: - repository: odpf/siren + repository: gotocompany/siren pullPolicy: Always tag: latest container: @@ -10,11 +10,35 @@ app: livenessProbe: httpGet: path: /ping - port: tcp + port: http readinessProbe: httpGet: path: /ping - port: tcp + port: http + ports: + - name: http + containerPort: 8080 + protocol: TCP + - name: grpc + containerPort: 8081 + protocol: TCP + volumeMounts: + - mountPath: /work-dir + name: workdir + initContainers: + - args: + [ + "wget -c https://github.com/goto/siren/releases/download/v0.6.9/siren-cortex_Linux_x86_64.tar.gz -O - | sudo tar -xz -C /work-dir/", + ] + command: ["/bin/sh", "-c"] + image: busybox:latest + imagePullPolicy: IfNotPresent + name: init-cortex + volumeMounts: + - mountPath: /work-dir + name: workdir + + podLabels: {} migration: enabled: true @@ -22,6 +46,17 @@ app: - server - migrate + service: + ports: + - port: 80 + name: http + targetPort: http + protocol: TCP + - port: 8081 + name: grpc + targetPort: grpc + protocol: TCP + ingress: enabled: true hosts: @@ -33,21 +68,34 @@ app: service: port: number: 80 + - host: siren-grpc.example.com + paths: + - path: / + pathType: ImplementationSpecific + backend: + service: + port: + number: 8081 config: - # SERVICE_PORT: 80 + # SERVICE_PORT: 8080 + SERVICE_GRPC_PORT: 8081 DB_DRIVER: postgres NOTIFICATION_QUEUE_KIND: postgres NOTIFICATION_MESSAGE_HANDLER_ENABLED: false NOTIFICATION_DLQ_HANDLER_ENABLED: false # PROVIDERS_CORTEX_WEBHOOK_BASE_API: http://localhost:8080/v1beta1/alerts/cortex - + secretConfig: DB_URL: postgres://postgres:@localhost:5432/siren?sslmode=disable SERVICE_ENCRYPTION_KEY: ____STRING_OF_32_CHARACTERS_____ + volumes: + - name: workdir + emptyDir: {} + notification-worker: image: - repository: odpf/siren + repository: gotocompany/siren pullPolicy: Always tag: latest container: @@ -58,19 +106,20 @@ notification-worker: livenessProbe: exec: command: - - /bin/sh - - -c - - ps -ef | grep siren | grep -v grep + - /bin/sh + - -c + - ps -ef | grep siren | grep -v grep initialDelaySeconds: 10 periodSeconds: 10 readinessProbe: exec: command: - - /bin/sh - - -c - - ps -ef | grep siren | grep -v grep + - /bin/sh + - -c + - ps -ef | grep siren | grep -v grep initialDelaySeconds: 10 periodSeconds: 10 + podLabels: {} migration: enabled: false @@ -83,14 +132,14 @@ notification-worker: NOTIFICATION_QUEUE_KIND: postgres NOTIFICATION_MESSAGE_HANDLER_ENABLED: false NOTIFICATION_DLQ_HANDLER_ENABLED: false - + secretConfig: DB_URL: postgres://postgres:@localhost:5432/siren?sslmode=disable SERVICE_ENCRYPTION_KEY: ____STRING_OF_32_CHARACTERS_____ notification-dlq-worker: image: - repository: odpf/siren + repository: gotocompany/siren pullPolicy: Always tag: latest container: @@ -101,26 +150,27 @@ notification-dlq-worker: livenessProbe: exec: command: - - /bin/sh - - -c - - ps -ef | grep siren | grep -v grep + - /bin/sh + - -c + - ps -ef | grep siren | grep -v grep initialDelaySeconds: 10 periodSeconds: 10 readinessProbe: exec: command: - - /bin/sh - - -c - - ps -ef | grep siren | grep -v grep + - /bin/sh + - -c + - ps -ef | grep siren | grep -v grep initialDelaySeconds: 10 periodSeconds: 10 + podLabels: {} migration: enabled: false ingress: enabled: false - + config: DB_DRIVER: postgres NOTIFICATION_QUEUE_KIND: postgres diff --git a/stable/stencil/Chart.lock b/stable/stencil/Chart.lock deleted file mode 100644 index 0cee54cf..00000000 --- a/stable/stencil/Chart.lock +++ /dev/null @@ -1,6 +0,0 @@ -dependencies: -- name: app - repository: https://odpf.github.io/charts/ - version: 0.2.1 -digest: sha256:7c3f11f696cd963558e25c7c40153c5b895fb0ea892cba55c4902e9f8ad1cc0a -generated: "2022-03-13T20:20:33.036613-07:00" diff --git a/stable/stencil/Chart.yaml b/stable/stencil/Chart.yaml index fd0380e6..2c2d2c01 100644 --- a/stable/stencil/Chart.yaml +++ b/stable/stencil/Chart.yaml @@ -26,4 +26,4 @@ appVersion: "0.2.0" dependencies: - name: app version: "0.2.1" - repository: "https://odpf.github.io/charts/" + repository: "https://goto.github.io/charts-legacy/" diff --git a/stable/stencil/values.yaml b/stable/stencil/values.yaml index 1bcfb50e..c30fb942 100644 --- a/stable/stencil/values.yaml +++ b/stable/stencil/values.yaml @@ -1,6 +1,6 @@ app: image: - repository: odpf/stencil + repository: gotocompany/stencil pullPolicy: Always tag: latest container: