-
Notifications
You must be signed in to change notification settings - Fork 15
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kafka: manager: List topics before creating them #316
Conversation
Updates the manager `CreateTopics` method to only issue the CreateTopics call if the topics don't exist in Kafka, preventing undesirable errors from being returned and logged. Signed-off-by: Marc Lopez Rubio <[email protected]>
kafka/topiccreator.go
Outdated
alterCfg = append(alterCfg, kadm.AlterConfig{Name: k, Value: v}) | ||
} | ||
alterResp, err := c.m.adminClient.AlterTopicConfigs(ctx, | ||
alterCfg, topicNames..., |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we only be altering topic configs for topics that we didn't just create? We'll get a lot of log spam otherwise won't we?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could, I didn't as to decouple the two... But I suppose we can. I can change it if you think it's significantly better. If it's just due to the logging, we could consolidate the info logging to a single line and use a topic slice instead of a single log line per topic altered.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if it's one log line vs. N, I'd rather not see an Info log every time this function is called if it's effectively a no-op. That aside, it would be good to minimise the number of network calls - that's a smaller issue though, considering how infrequently this should be called.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've updated the code and the test a bit in 8acf0af. Let me know what you think.
Signed-off-by: Marc Lopez Rubio <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, thanks!
Co-authored-by: Andrew Wilkins <[email protected]>
Signed-off-by: Marc Lopez Rubio <[email protected]>
I've opened #318 to investigate. |
Updates the manager
CreateTopics
method to only issue the CreateTopics call if the topics don't exist in Kafka, preventing undesirable errors from being returned and logged.