Kafka exporters: client has run out of available brokers to talk to: 3 errors occurred:

[xiaoyao2246]

Component(s)

exporter/kafka

Describe the issue you're reporting

My Kubernetes version is 1.20.0, and I use the following configuration to deploy the Collector:

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-collector-gd-config
  namespace: test
data:
  config.yaml: |-
    receivers:
      #jaeger:
      #  protocols:
      #    thrift_http:
      #      endpoint: 0.0.0.0:4316
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
    processors:
      batch:
        send_batch_size: 500
        send_batch_max_size: 500
      resource:
        attributes:
          - key: from-collector
            value: xxxx
            action: insert
    exporters:
      logging:
        verbosity: detailed
      kafka:
        brokers:
          - xxxx.xxxx.xxxx.xxxx:9092
          - xxxx.xxxx.xxxx.xxxx:9092
          - xxxx.xxxx.xxxx.xxxx:9092
        topic: otlp_trace_fat
        partition_traces_by_id: true
        protocol_version: 2.0.0
    service:
      pipelines:
        traces:
          receivers: [otlp]
          processors: [batch, resource]
          exporters: [logging, kafka]

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: otel-collector-gd
  namespace: test
  labels:
    app: opentelemetry
    component: otel-collector-gd
spec:
  selector:
    matchLabels:
      app: opentelemetry
      component: otel-collector-gd
  template:
    metadata:
      labels:
        app: opentelemetry
        component: otel-collector-gd
    spec:
      containers:
        - name: otel-collector-gd
          image: otel/opentelemetry-collector-contrib:0.95.0
          resources:
            limits:
              cpu: 1000m
              memory: 2048Mi
          volumeMounts:
            - mountPath: /var/log
              name: varlog
              readOnly: true
            - mountPath: /var/lib/docker/containers
              name: varlibdockercontainers
              readOnly: true
            - mountPath: /etc/otelcol-contrib/config.yaml
              name: data
              subPath: config.yaml
              readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
        - name: varlog
          hostPath:
            path: /var/log
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: data
          configMap:
            name: otel-collector-gd-config

---
apiVersion: v1
kind: Service
metadata:
  name: otel-collector-gd
  namespace: test
  labels:
    app: opentelemetry
    component: otel-collector-gd
spec:
  ports:
    - name: otlp-grpc
      port: 4317
      protocol: TCP
      targetPort: 4317
    - name: otlp-http
      port: 4318
      protocol: TCP
      targetPort: 4318
  selector:
    component: otel-collector-gd

An error is reported when the Collector is started.

2024-12-25T03:34:19.043Z	info	[email protected]/telemetry.go:55	Setting up own telemetry...
2024-12-25T03:34:19.043Z	info	[email protected]/telemetry.go:97	Serving metrics	{"address": ":8888", "level": "Basic"}
2024-12-25T03:34:19.044Z	info	[email protected]/exporter.go:275	Deprecated component. Will be removed in future releases.	{"kind": "exporter", "data_type": "traces", "name": "logging"}
Error: failed to build pipelines: failed to create "kafka" exporter for data type "traces": kafka: client has run out of available brokers to talk to: 3 errors occurred:
	* EOF
	* EOF
	* EOF

2024/12/25 03:34:19 collector server run finished with error: failed to build pipelines: failed to create "kafka" exporter for data type "traces": kafka: client has run out of available brokers to talk to: 3 errors occurred:
	* EOF
	* EOF
	* EOF

The network between the server and Kafka is normal. The Collector of version v.0.40.0 can run properly, but higher - versioned images don't work. Are there any troubleshooting methods, please?

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[xiaoyao2246]

The problem has been found. The reason is that the Kafka version used is version 1.0. Therefore, the following attributes need to be modified.
protocol_version: 1.0.0