cannot deploy custom images for KafkaConnect from private image repo using Strimzi operator

[igorpalant]

I have built a custom kafka connect image that includes our required connectors. I am trying to deploy kafkaconnect resource using custom image that is located in our private repo. It seems that instructions imagePullPolicy and imagePullSecret do not get processed by strimzi and image cannot be downloaded from our private repo. I might be missing something. I could not find a good place to look, other than CRDs. Please help

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnect
metadata:
  name: oc-platform-kafka-connect
  annotations:
    strimzi.io/use-connector-resources: "true"
spec:
  image: ourrepo/mongodb-kafka-connector:1.0.1
  imagePullPolicy: Always
  imagePullSecret: our-secret
  replicas: 1
  bootstrapServers: platform-kafka-kafka-bootstrap:9093
  tls:
    trustedCertificates:
      - secretName: platform-kafka-cluster-ca-cert
        certificate: ca.crt
  config:
    group.id: connect-cluster
    config.storage.replication.factor: 1
    offset.storage.replication.factor: 1
    status.storage.replication.factor: 1
    offset.storage.topic: connect-cluster-offsets
    config.storage.topic: connect-cluster-configs
    status.storage.topic: connect-cluster-status

[scholzj]

You cannot just add random fields and expect them to magically work. You can either configure image pull secrets at the Cluster operator level or in the PodTemplate section.

[mateuszkwiatkowski]

Hi @scholzj,
Is it possible to configure imagePullPolicy for Kafka Connect pods? We're building Connect images in our CI but they are not pulled automatically by pods.

[scholzj]

I think imagePullPolicy can be configured only cluster-wide in the Cluster Operator - with the STRIMZI_IMAGE_PULL_POLICY environment variable.

[mateuszkwiatkowski]

Thank you for prompt response!

[jtroxel]

So one can't set STRIMZI_IMAGE_PULL_POLICY in the spec for KafkaConnect like the OP ^^?

I have a similar situation to the above, setting up Strimzi locally in Minikube for now. It is not clean at all to me where/how one uses the STRIMZI_IMAGE_PULL_POLICY env variable so that I can reference a custom image in the KafkaConnect spec.

[scholzj]

The Image pull policy is configured with the STRIMZI_IMAGE_PULL_POLICY in the Strimzi Cluster operator Deployment.

[jtroxel]

Right, I got that. I was hoping for an example that shows how that or whatever env variable can be set for the deployment.

I have been leveraging Strizmi Because it's simplifies and abstracts Kubernetes details, So what I have had to do is helm install the operator, and then use kubectl to Apply the Kafka and Kafka connect files. so where in that process would I add in this variable?

[scholzj]

If you used Helm, you should pass it as a parameter when installing the Helm chart. But I'm not a Helm user, so I will not give you any detailed instructions I'm afraid. If you would install it through the YAML files, you can just edit the Deployment with kubectl edit deployment ... or edit the YAML file and kubectl apply it again. The Helm Chart has some parameters for the env vars, but not sure about the names. It will be covered in the Helm Chart README.

---

TBH, Strimzi does not aim in any way to abstract Kubernetes details. If anything, Strimzi aims to abstract Kafka details from Kubernetes users. So you should have some Kubernetes knowledge to use Strimzi. How much of it will depend what are you trying to do. You might need to know more for production and less for some development stuff.

[jtroxel]

fair enough, thank you. I guess in my case abstracting the kafka details also allowed me to just barely follow some of the guides 😄