10.0.2.2 Kafka with new go client

[golangcyberdude]

Hi,

Is it possible for the latest go client to connect to Kafka 10.0.2.2 Zookeeper for client connections?

On older versions of Kafka this may resolve various connectivity issues in the client consumer,
instead of connecting to the brokers directly.

This may be required for older Kafka server we have in Production.

I am able to connect to zookeeper as consumer via CLI:

./kafka-console-consumer.sh --zookeeper 9.9.9.9:2181 --topic redact

I do have broker client connections going but can not connect to zookeeper in the client.

        consumer, err := kafka.NewConsumer(&kafka.ConfigMap{
                "group.id":                            "KafkaConsumerGroupID",
                "bootstrap.servers":                   "9.9.9.9:2181",  // redacted 
                "security.protocol":                   "plaintext",                                              
                "api.version.request":                 "true",  // tried false too
                "debug":                               "all", //security,broker",
                "max.poll.interval.ms":                300000,
                "broker.version.fallback":             "0.8.1",  // tried various things here.
                "auto.offset.reset":                   "beginning"})

%6|1644515385.171|FAIL|rdkafka#consumer-1| [thrd:9.9.9.9:2181/bootstrap]: 9.9.9.9:2181/bootstrap: Disconnected while requesting ApiVersion: might be caused by incorrect security.protocol configuration (connecting to a SSL listener?) or broker version is < 0.10 (see api.version.request) (after 0ms in state APIVERSION_QUERY)

is this a work-around? https://github.com/go-zookeeper/zk looking into it.

Thanks

[edenhill]

Clients should not connect to Zookeeper, and there's work to remove zookeeper completely from Kafka, so this seems like the wrong path to go down.

[golangcyberdude]

Unfortunately, we have a Kafka server at 0.10.2.2 that can not be changed at this time.

Is there any older go client library that would connect to zookeeper in a similar way?

./kafka-console-consumer.sh --zookeeper 9.9.9.9:2181 --topic redact

Perhaps this client might work ? or is this just not going to work at all?

https://github.com/confluentinc/confluent-kafka-go/tree/v0.11.6/kafka

Kafka server 0.10.2.2 Released July 2nd, 2018
go client v0.11.6 Latest commit on Oct 23, 2018

Thank You for your help.