WIP: GEFEST-856 Конфигурация TLS для Kafka в чарте Keys

charts/keys/Chart.yaml

@@ -4,7 +4,7 @@ type: application
 description: A Helm chart for Kubernetes to deploy API Keys service
 
 version: 1.31.0
-appVersion: 1.89.0
+appVersion: 1.102.3
 
 maintainers:
 - name: 2gis

charts/keys/README.md

@@ -31,7 +31,7 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 | `imagePullSecrets`         | Kubernetes image pull secrets.    | `[]`                           |
 | `imagePullPolicy`          | Pull policy.                      | `IfNotPresent`                 |
 | `backend.image.repository` | Backend service image repository. | `2gis-on-premise/keys-backend` |
-| `backend.image.tag`        | Backend service image tag.        | `1.89.0`                       |
+| `backend.image.tag`        | Backend service image tag.        | `1.102.3`                      |
 | `admin.image.repository`   | Admin service image repository.   | `2gis-on-premise/keys-ui`      |
 | `admin.image.tag`          | Admin service image tag.          | `0.10.3`                       |
 | `redis.image.repository`   | Redis image repository.           | `2gis-on-premise/keys-redis`   |
@@ -226,15 +226,21 @@ See the [documentation](https://docs.2gis.com/en/on-premise/keys) to learn about
 
 ### Kafka settings
 
-| Name                                  | Description                                                                                                                                                | Value  |
-| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ------ |
-| `kafka.audit`                         | **Settings for sending audit messages.**                                                                                                                   |        |
-| `kafka.audit.bootstrapServers`        | Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093').                            | `""`   |
-| `kafka.audit.username`                | Username for authorization (SASL/PLAINTEXT SHA-512).                                                                                                       | `""`   |
-| `kafka.audit.password`                | Password for authorization (SASL/PLAINTEXT SHA-512).                                                                                                       | `""`   |
-| `kafka.audit.topic`                   | Topic to produce audit messages.                                                                                                                           | `""`   |
-| `kafka.audit.produce.retryCount`      | Number of retries to produce a message.                                                                                                                    | `5`    |
-| `kafka.audit.produce.idempotentWrite` | Flag to enable/disable [idempotent write](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#enable-idempotence). | `true` |
+| Name                                    | Description                                                                                                                                                | Value       |
+| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------- |
+| `kafka.bootstrapServers`                | Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093'). **Required**               | `""`        |
+| `kafka.securityProtocol`                | Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Default: PLAINTEXT.                                 | `PLAINTEXT` |
+| `kafka.saslMechanism`                   | Authentication mechanism when security_protocol is configured for SASL_PLAINTEXT or SASL_SSL. Valid values are: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512.       | `PLAIN`     |
+| `kafka.username`                        | Username for authorization (SASL).                                                                                                                         | `""`        |
+| `kafka.password`                        | Password for authorization (SASL).                                                                                                                         | `""`        |
+| `kafka.tls.skipServerCertificateVerify` | Controls whether a client verifies the server's certificate chain and host name.                                                                           | `false`     |
+| `kafka.tls.rootCert`                    | Root certificate.                                                                                                                                          | `""`        |
+| `kafka.tls.cert`                        | Client certificate.                                                                                                                                        | `""`        |
+| `kafka.tls.key`                         | Client key.                                                                                                                                                | `""`        |
+| `kafka.audit`                           | **Settings for sending audit messages.**                                                                                                                   |             |
+| `kafka.audit.topic`                     | Topic to produce audit messages. **Required**                                                                                                              | `""`        |
+| `kafka.audit.produce.retryCount`        | Number of retries to produce a message.                                                                                                                    | `5`         |
+| `kafka.audit.produce.idempotentWrite`   | Flag to enable/disable [idempotent write](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#enable-idempotence). | `true`      |
 
 ### LDAP connection settings
 

charts/keys/templates/api/deployment.yaml

@@ -58,7 +58,6 @@ spec:
             {{- include "keys.env.db.deploys" . | nindent 12 }}
             {{- include "keys.env.redis" . | nindent 12 }}
             {{- include "keys.env.auth" . | nindent 12 }}
-            {{- include "keys.env.kafka.audit" . | nindent 12 }}
             {{- if .Values.customCAs.bundle }}
             {{- include "keys.env.custom.ca.path" . | nindent 12 }}
             {{- end }}

charts/keys/templates/dispatcher/deployment.yaml

@@ -36,6 +36,26 @@ spec:
         {{- toYaml . | nindent 8 }}
         {{- end }}
     spec:
+      {{ $kafkaSslEnabled := include "kafka.ssl.enabled" .}}
+      {{- if $kafkaSslEnabled }}
+      initContainers:
+        - name: copy-kafka-certs
+          image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag | default .Chart.AppVersion }}
+          imagePullPolicy: {{ .Values.imagePullPolicy }}
+          resources:
+            {{- toYaml .Values.api.resources | nindent 12 }}
+          command:
+            - /bin/sh
+            - -ec
+            - |
+              cp /tmp/certs/* /etc/2gis/secret/kafka/
+              chmod 600 /etc/2gis/secret/kafka/client.key
+          volumeMounts:
+            - name: {{ include "keys.kafka-raw.name" . | quote }}
+              mountPath: /tmp/certs
+            - name: {{ include "keys.kafka.name" . | quote }}
+              mountPath: /etc/2gis/secret/kafka
+      {{- end }}
       containers:
         - name: dispatcher
           image: {{ required "A valid .Values.dgctlDockerRegistry entry required" .Values.dgctlDockerRegistry }}/{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}
@@ -51,13 +71,28 @@ spec:
             {{- if .Values.customCAs.bundle }}
             {{- include "keys.env.custom.ca.path" . | nindent 12 }}
             {{- end }}
-          {{- if .Values.customCAs.bundle }}
+          {{- if or .Values.customCAs.bundle $kafkaSslEnabled }}
           volumeMounts:
+            {{- if .Values.customCAs.bundle }}
             {{- include "keys.custom.ca.volumeMounts" . | nindent 12 }}
+            {{- end }}
+            {{- if $kafkaSslEnabled }}
+            - name: {{ include "keys.kafka.name" . | quote }}
+              mountPath: /etc/2gis/secret/kafka
+            {{- end }}
           {{- end }}
-      {{- if .Values.customCAs.bundle }}
+      {{- if or .Values.customCAs.bundle $kafkaSslEnabled }}
       volumes:
+        {{- if .Values.customCAs.bundle }}
         {{- include "keys.custom.ca.deploys.volumes" . | nindent 8 }}
+        {{- end }}
+        {{- if $kafkaSslEnabled }}
+        - name: {{ include "keys.kafka-raw.name" . | quote }}
+          secret:
+            secretName: {{ include "keys.kafka.name" . | quote }}
+        - name: {{ include "keys.kafka.name" . | quote }}
+          emptyDir: {}
+        {{- end }}
       {{- end }}
       {{- with .Values.dispatcher.nodeSelector }}
       nodeSelector:

charts/keys/templates/helpers.tpl

@@ -42,6 +42,22 @@
 {{ include "keys.name" . }}-jobs
 {{- end }}
 
+{{- /*
+Name for kafka intermediate volume for copy secrets
+*/ -}}
+
+{{- define "keys.kafka-raw.name" -}}
+{{- printf "%s-kafka-raw" (include "keys.name" .) -}}
+{{- end }}
+
+{{- /*
+Name for kafka secret and volume
+*/ -}}
+
+{{- define "keys.kafka.name" -}}
+{{- printf "%s-kafka" (include "keys.name" .) -}}
+{{- end }}
+
 {{- define "keys.selectorLabels" -}}
 app.kubernetes.io/name: {{ .Chart.Name }}
 app.kubernetes.io/instance: {{ .Release.Name }}
@@ -199,10 +215,10 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
   value: "{{ .Values.postgres.rw.schema }}"
 - name: KEYS_DB_RW_USERNAME
   value: "{{ required "A valid .Values.postgres.rw.username required" .Values.postgres.rw.username }}"
-{{- end }}
+{{- end -}}
 
 {{- define "keys.env.db.deploys" -}}
-{{ include "keys.env.db" . }}
+{{- include "keys.env.db" . }}
 - name: KEYS_DB_RO_PASSWORD
   valueFrom:
     secretKeyRef:
@@ -213,10 +229,10 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
     secretKeyRef:
       name: {{ include "keys.secret.deploys.name" . }}
       key: dbRWPassword
-{{- end }}
+{{- end -}}
 
 {{- define "keys.env.db.jobs" -}}
-{{ include "keys.env.db" . }}
+{{- include "keys.env.db" . }}
 - name: KEYS_DB_RO_PASSWORD
   valueFrom:
     secretKeyRef:
@@ -227,18 +243,18 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
     secretKeyRef:
       name: {{ include "keys.secret.jobs.name" . }}
       key: dbRWPassword
-{{- end }}
+{{- end -}}
 
 {{- define "keys.env.redis" -}}
-{{- if .Values.redis.useExternalRedis }}
+{{- if .Values.redis.useExternalRedis -}}
 - name: KEYS_REDIS_HOST
   value: "{{ .Values.redis.host }}"
 - name: KEYS_REDIS_DB
   value: "{{ .Values.redis.db }}"
-{{- else  }}
+{{- else -}}
 - name: KEYS_REDIS_HOST
   value: "{{ include "keys.redis.name" . }}"
-{{- end  }}
+{{- end }}
 - name: KEYS_REDIS_PORT
   value: "{{ .Values.redis.port }}"
 {{- if .Values.redis.password }}
@@ -247,11 +263,11 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
     secretKeyRef:
       name: {{ include "keys.secret.deploys.name" . }}
       key: redisPassword
-{{- end }}
-{{- end }}
+{{- end -}}
+{{- end -}}
 
 {{- define "keys.env.auth" -}}
-{{- if .Values.api.adminUsers }}
+{{- if .Values.api.adminUsers -}}
 - name: KEYS_ADMIN_USERS
   valueFrom:
     secretKeyRef:
@@ -346,13 +362,32 @@ app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
 
 {{- define "keys.env.kafka.audit" -}}
 - name: KEYS_KAFKA_AUDIT_BROKERS
-  value: "{{ .Values.kafka.audit.bootstrapServers }}"
+  value: "{{ required "A valid .Values.kafka.bootstrapServers entry required" .Values.kafka.bootstrapServers }}"
 - name: KEYS_KAFKA_AUDIT_USERNAME
-  value: "{{ .Values.kafka.audit.username }}"
+  value: "{{ .Values.kafka.username }}"
+{{- if .Values.kafka.password }}
 - name: KEYS_KAFKA_AUDIT_PASSWORD
-  value: "{{ .Values.kafka.audit.password }}"
+  valueFrom:
+    secretKeyRef:
+      name: {{ include "keys.kafka.name" . }}
+      key: password
+{{- end }}
+- name: KEYS_KAFKA_AUDIT_SECURITY_PROTOCOL
+  value: "{{ .Values.kafka.securityProtocol }}"
+- name: KEYS_KAFKA_AUDIT_SASL_MECHANISM
+  value: "{{ .Values.kafka.saslMechanism }}"
+{{- if (include "kafka.ssl.enabled" .) }}
+- name: KEYS_KAFKA_AUDIT_TLS_SKIP_SERVER_CERTIFICATE_VERIFY
+  value: "{{ .Values.kafka.tls.skipServerCertificateVerify }}"
+- name: KEYS_KAFKA_AUDIT_TLS_CLIENT_CERTIFICATE_PATH
+  value: "/etc/2gis/secret/kafka/client.crt"
+- name: KEYS_KAFKA_AUDIT_TLS_CLIENT_KEY_PATH
+  value: "/etc/2gis/secret/kafka/client.key"
+- name: KEYS_KAFKA_AUDIT_TLS_CA_CERT_PATH
+  value: "/etc/2gis/secret/kafka/ca.crt"
+{{- end }}
 - name: KEYS_KAFKA_AUDIT_TOPIC
-  value: "{{ .Values.kafka.audit.topic }}"
+  value: "{{ required "A valid .Values.kafka.audit.topic entry required" .Values.kafka.audit.topic }}"
 - name: KEYS_KAFKA_AUDIT_PRODUCE_RETRY_COUNT
   value: "{{ .Values.kafka.audit.produce.retryCount }}"
 - name: KEYS_KAFKA_AUDIT_PRODUCE_IDEMPOTENT_WRITE
@@ -424,3 +459,10 @@ Return the appropriate apiVersion for Horizontal Pod Autoscaler.
 {{- define "keys.configmap.deploys.name" -}}
 {{ include "keys.name" . }}-configmap-deploys
 {{- end -}}
+
+{{- define "kafka.ssl.enabled" }}
+{{- $securityProtocol := index .Values.kafka.securityProtocol -}}
+{{- $isEnabled := or (eq $securityProtocol "SSL") (eq $securityProtocol "SASL_SSL") -}}
+{{/* Converting bool to "thruthy" string cause "include" can only return string. */}}
+{{- ternary "true" "" $isEnabled }}
+{{- end -}}

charts/keys/templates/secret-kafka.yaml

@@ -0,0 +1,20 @@
+{{- $sslEnabled := include "kafka.ssl.enabled" . }}
+{{- $password := .Values.kafka.password }}
+{{- if or ($sslEnabled) (not (empty $password)) }}
+apiVersion: v1
+kind: Secret
+metadata:
+  name: {{ include "keys.kafka.name" . | quote }}
+  labels:
+    {{- include "keys.labels" . | nindent 4 }}
+type: Opaque
+data:
+  {{- if $sslEnabled }}
+  client.crt: {{ required "A valid kafka.tls.cert entry required" .Values.kafka.tls.cert | b64enc | quote }}
+  client.key: {{ required "A valid kafka.tls.key entry required" .Values.kafka.tls.key | b64enc | quote }}
+  ca.crt: {{ required "A valid kafka.tls.rootCert entry required" .Values.kafka.tls.rootCert | b64enc | quote }}
+  {{- end }}
+  {{- with $password }}
+  password: {{ $password | b64enc }}
+  {{- end }}
+{{- end }}

charts/keys/values.yaml

@@ -31,7 +31,7 @@ featureFlags:
 backend:
   image:
     repository: 2gis-on-premise/keys-backend
-    tag: 1.89.0
+    tag: 1.102.3
 
 # @section Admin service settings
 
@@ -538,19 +538,34 @@ postgres:
 
 kafka:
 
+  # @param kafka.bootstrapServers Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093'). **Required**
+  # @param kafka.securityProtocol Protocol used to communicate with brokers. Valid values are: PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL. Default: PLAINTEXT.
+  # @param kafka.saslMechanism Authentication mechanism when security_protocol is configured for SASL_PLAINTEXT or SASL_SSL. Valid values are: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512.
+  # @param kafka.username Username for authorization (SASL).
+  # @param kafka.password Password for authorization (SASL).
+  # @param kafka.tls.skipServerCertificateVerify Controls whether a client verifies the server's certificate chain and host name.
+  # @param kafka.tls.rootCert Root certificate.
+  # @param kafka.tls.cert Client certificate.
+  # @param kafka.tls.key Client key.
+
+  bootstrapServers: ''
+  securityProtocol: PLAINTEXT
+  saslMechanism: PLAIN
+  username: ''
+  password: ''
+  tls:
+    skipServerCertificateVerify: false
+    rootCert: ''
+    cert: ''
+    key: ''
+
   # @extra kafka.audit **Settings for sending audit messages.**
 
-  # @param kafka.audit.bootstrapServers Comma-separated list of host and port pairs that are the addresses of the Kafka brokers (e.g. 'localhost:9092,localhost:9093').
-  # @param kafka.audit.username Username for authorization (SASL/PLAINTEXT SHA-512).
-  # @param kafka.audit.password Password for authorization (SASL/PLAINTEXT SHA-512).
-  # @param kafka.audit.topic Topic to produce audit messages.
+  # @param kafka.audit.topic Topic to produce audit messages. **Required**
   # @param kafka.audit.produce.retryCount Number of retries to produce a message.
   # @param kafka.audit.produce.idempotentWrite Flag to enable/disable [idempotent write](https://docs.confluent.io/platform/current/installation/configuration/producer-configs.html#enable-idempotence).
 
   audit:
-    bootstrapServers: ''
-    username: ''
-    password: ''
     topic: ''
     produce:
       retryCount: 5