librdkafka 1.6.0 , C++ API producer, producing immediately after topic creation SOMETIMES result in errors (in RdKafka::Producer::produce call) #3977
aKumara123
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
librdkafka version => 1.6.0
topic.metadata.propagation.max.ms=60000
Topic auto creation NOT used (in our use case different topics need to have different partition counts). OS = Red-hat Linux 7.9
Message is sent(RdKafka::Producer::produce) immediately after successfully creating the topic (rd_kafka_CreateTopics => rd_kafka_queue_poll => if NOT timed out, check errors with rd_kafka_event_error and rd_kafka_event_error+rd_kafka_CreateTopics_result_topics+rd_kafka_topic_result_error => RD_KAFKA_RESP_ERR_NO_ERROR is taken as success)
Same RdKafka::Producer used to create topics and Produce. Both calls made in in same thread.
But the produce call/delivery fails some times with below errors(other times it is successful),
Produce call fail : ERR__UNKNOWN_TOPIC, ERR__FATAL
Delivery fail (Callback) : ERR__DESTROY, ERR__UNKNOWN_PARTITION
What is the recommended solution here ?
Add a sleep after each topic creation ?
OR
any other ?
[same question posted on Stack overflow too, but no response received yet 2022/09/13]
Beta Was this translation helpful? Give feedback.
All reactions