KafkaConnect Reconciliation encounter: disallow-capabilities-strict:\n require-drop-all: 'validation failure: Containers must drop ALL capabilities

[KevinMeng4662]

Hi Team,

There is one issue as below for KafkaConnect Reconciliation.

{"instant":{"epochSecond":1726618252,"nanoOfSecond":192783309},"thread":"StrimziPodSetController","level":"ERROR","loggerName":"io.strimzi.operator.cluster.operator.assembly.StrimziPodSetController","marker":{"name":"StrimziPodSet(XXXX/XXXX-connect)"},"message":"Reconciliation #1080(watch) StrimziPodSet(XXXX/XXXX-connect): StrimziPodSet XXXX-connect in namespace XXXX reconciliation failed","thrown":{"commonElementCount":0,"localizedMessage":"Failure executing: POST at: https://api.025qaauto01.hxmnonprod.internal.live.k8s.ondemand.com:443/api/v1/namespaces/XXXX/pods. Message: admission webhook "validate.kyverno.svc-fail" denied the request: \n\nresource Pod/XXXX/XXXX-connect-0 was blocked due to the following policies \n\ndisallow-capabilities-strict:\n require-drop-all: 'validation failure: Containers must drop ALL capabilities.'\n. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validate.kyverno.svc-fail" denied the request: \n\nresource Pod/XXXX/XXXX-connect-0 was blocked due to the following policies \n\ndisallow-capabilities-strict:\n require-drop-all: 'validation failure: Containers must drop ALL capabilities.'\n, metadata=ListMeta(_continue=null, remainingItemCount=null, resourceVersion=null, selfLink=null, additionalProperties={}), reason=null, status=Failure, additionalProperties={}).","message":"Failure executing: POST at: https://api.025qaauto01.hxmnonprod.internal.live.k8s.ondemand.com:443/api/v1/namespaces/XXXX/pods. Message: admission webhook "validate.kyverno.svc-fail" denied the request: \n\nresource Pod/XXXX/XXXX-connect-0 was blocked due to the following policies \n\ndisallow-capabilities-strict:\n require-drop-all: 'validation failure: Containers must drop ALL capabilities.'\n. Received status: Status(apiVersion=v1, code=400, details=null, kind=Status, message=admission webhook "validate.kyverno.svc-fail" denied the request: \n\nresource Pod/XXXX/XXXX-connect-0 was blocked due to the following policies \n\ndisallow-capabilities-strict:\n require-drop-all: 'validation failure: Containers must drop ALL capabilities.'\n,

The KafkaConnect configuration as below, and I think the securityContext configuration should resolve current issue, but it does not work, could you help on that?

connectContainer:
    securityContext:
        readOnlyRootFilesystem: true
        runAsNonRoot: true
        capabilities:
          add: [ ]
          drop:
            - ALL

---------kafka-connect.yaml-------

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: XXXX
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  version: 3.7.1
  replicas: 4
  bootstrapServers: bootstrapServers
  authentication:
    type: plain
    username: username
    passwordSecret:
      secretName: onekafka-connect-secret
      password: password
  tls:
    trustedCertificates:
      []
  resources:
    requests:
      cpu: 500m
      memory: 1Gi
    limits:
      cpu: 1
      memory: 2Gi
  readinessProbe:
    initialDelaySeconds: 15
    timeoutSeconds: 5
  livenessProbe:
    initialDelaySeconds: 15
    timeoutSeconds: 5
  jvmOptions:
    -Xms: 512m
    -Xmx: 1g
    javaSystemProperties:
      - name: ssl.KeyManagerFactory.algorithm
        value: SunX509
      - name: javax.net.ssl.keyStoreType
        value: PKCS12
  rack:
    topologyKey: topology.kubernetes.io/zone
  metricsConfig:
    type: jmxPrometheusExporter
    valueFrom:
      configMapKeyRef:
        name: configs
        key: connect-metrics.yaml
  config:
    security.protocol: SSL
    group.id: id
    offset.storage.topic: offsets
    config.storage.topic: configs
    status.storage.topic: status
    config.storage.cleanup.policy: compact
    offset.storage.cleanup.policy: compact
    status.storage.cleanup.policy: compact
    # -1 means it will use the default replication factor configured in the broker
    config.storage.replication.factor: -1
    offset.storage.replication.factor: -1
    status.storage.replication.factor: -1
    offset.storage.partitions: 25
    key.converter: org.apache.kafka.connect.converters.ByteArrayConverter
    key.converter.schemas.enable: false
    value.converter: org.apache.kafka.connect.converters.ByteArrayConverter
    value.converter.schemas.enable: false
    header.converter: org.apache.kafka.connect.converters.ByteArrayConverter
    header.converter.schemas.enable: false
    config.providers: secrets,configmaps,env
    config.providers.secrets.class: io.strimzi.kafka.KubernetesSecretConfigProvider
    config.providers.configmaps.class: io.strimzi.kafka.KubernetesConfigMapConfigProvider
    config.providers.env.class: org.apache.kafka.common.config.provider.EnvVarConfigProvider
  template:
    pod:
      metadata:
        labels:
          app: XXXX
          app.kubernetes.io/name: XXXX
        annotations:
          metrics.dynatrace.com/port: '9404'
      imagePullSecrets:
        - name: sf-docker-staging-cicd-regcred
      securityContext:
        seccompProfile:
          type: RuntimeDefault
        runAsUser: 65536
        runAsNonRoot: true
        fsGroup: 65536
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: strimzi.io/name
                    operator: In
                    values:
                      - XXXX
              topologyKey: "kubernetes.io/hostname"
    connectContainer:
      securityContext:
        readOnlyRootFilesystem: true
        runAsNonRoot: true
        capabilities:
          add: [ ]
          drop:
            - ALL
      env:
        - name: KAFKA_OPTS
          value: "-Djavax.net.ssl.keyStorePassword=test"
  image: kafkaconnect-svc

Thanks.

[scholzj]

Unfortunately, the error message about the issue is not completely clear. So you might need to check with whoever configured that to tell you what exactly the issue is.

But with rack awareness enabled, you might need to configure the security context also for the init container used to gather the rack information. So that might be something worth trying.

[KevinMeng4662]

Thanks, it working fine