Listener specific perPodService

[abiodunakandewp]

Hi,

Looking for some guidance on a problem we're encountering with having both loadbalancer and route listeners within our kafka cluster.

Our current setup is:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: kafka-node-pool
  namespace: kafka
  labels:
    strimzi.io/cluster: kafka-cluster
spec:
  replicas: 1
...
  template:
    perPodService:
      metadata:
        annotations:
          service.beta.kubernetes.io/aws-load-balancer-internal: "true"
          service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.0/8
          service.beta.kubernetes.io/aws-load-balancer-subnets: ${SUBNET}
...
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: kafka-cluster
  namespace: kafka
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
...
  kafka:
    listeners:
    - name: externaltls
      port: 9093
      type: loadbalancer
      tls: true
      authentication:
        type: tls
      configuration:
        bootstrap:
          annotations:
            service.beta.kubernetes.io/aws-load-balancer-internal: "true"
            service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.0/8
            service.beta.kubernetes.io/aws-load-balancer-subnets: ${SUBNET}
    - name: route
       port: 9094
       tls: true
       type: route

This configuration currently fails because of the route listener. The service related to the route listener fails to create with error metadata.annotations[service.beta.kubernetes.io/load-balancer-source-ranges], message=Forbidden: may only be used when type is ''LoadBalancer'', reason=FieldValueForbidden,.

We were wondering if we're missing some option to define a perPodService type within the listener itself.

For example:

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
...
listeners:
- name: externaltls
  port: 9093
  type: loadBalancer
  perPodService:
    annotations:
     service.beta.kubernetes.io/aws-load-balancer-internal: "true"
     service.beta.kubernetes.io/load-balancer-source-ranges: 10.0.0.0/8
     service.beta.kubernetes.io/aws-load-balancer-subnets: ${SUBNET}
- name: route
  port: 9094
  tls: true
  type: route
  perPodService:
      annotations:
         <route-specific-annotation>: foo

We would like to avoid defining the annotations within spec.kafka.listeners[*].configuration.brokers[*] because when we scale the replicas within the node pool, we are then required to add another broker to the configuration.

[scholzj]

You can define the annotations on a per-listener or per-broker basis like this: https://strimzi.io/docs/operators/latest/full/configuring.html#property-listener-config-annotations-reference

[abiodunakandewp]

Hi,

Thanks for your reply.

We'd like to avoid this because that would mean when we scale up the kafkaNodePool.spec.replicas, we then have to add a broker entry within all of our listeners.

We want the configuration of all brokers associated with a listener to be uniform so it is nice to define it once within a perPodService type field.

[scholzj]

We'd like to avoid this because that would mean when we scale up the kafkaNodePool.spec.replicas, we then have to add a broker entry within all of our listeners.

Right. But as of now, I'm afraid this is the only way to do it. You can add brokers in advance there if it helps.

[abiodunakandewp]

Thanks,

Could we open up a feature request for perPodService to be added to spec.kafka.listeners[*].configuration?

[scholzj]

You can of course open an enhancement issue. That said - I do not think the API you suggested is a good fit as it is completely incompatible with the general API structure we use. I'm personally also not convinced that it is worth the effort to have a third way to manage the annotation when there is a relatively easy workaround (and while the scaling aspect might be a complication, Kafka cluster is not something where you are scaling by hundreds of new nodes every day).