kafka: manager: List topics before creating them #316
Conversation
Updates the manager `CreateTopics` method to only issue the CreateTopics call if the topics don't exist in Kafka, preventing undesirable errors from being returned and logged. Signed-off-by: Marc Lopez Rubio <[email protected]>
kafka/topiccreator.go
Outdated
| alterCfg = append(alterCfg, kadm.AlterConfig{Name: k, Value: v}) | ||
| } | ||
| alterResp, err := c.m.adminClient.AlterTopicConfigs(ctx, | ||
| alterCfg, topicNames..., |
There was a problem hiding this comment.
Should we only be altering topic configs for topics that we didn't just create? We'll get a lot of log spam otherwise won't we?
There was a problem hiding this comment.
We could, I didn't as to decouple the two... But I suppose we can. I can change it if you think it's significantly better. If it's just due to the logging, we could consolidate the info logging to a single line and use a topic slice instead of a single log line per topic altered.
There was a problem hiding this comment.
Even if it's one log line vs. N, I'd rather not see an Info log every time this function is called if it's effectively a no-op. That aside, it would be good to minimise the number of network calls - that's a smaller issue though, considering how infrequently this should be called.
There was a problem hiding this comment.
I've updated the code and the test a bit in 8acf0af. Let me know what you think.
Signed-off-by: Marc Lopez Rubio <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
Co-authored-by: Andrew Wilkins <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
|
I've opened #318 to investigate. |
Updates the manager
CreateTopicsmethod to only issue the CreateTopics call if the topics don't exist in Kafka, preventing undesirable errors from being returned and logged.