-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Safe recreation of topics #1739
Comments
Hey @szczygiel-m , is this up for grabs? |
Hi, sure 😄 Assigned |
hey it may be too late for hacktoberfest but I can take a look at this if for a couple of days, you can assign it to me if you want |
Looking at this, high level proposal
enhancements
I'll have a look at this next and see if I can build a concept PR |
Looks like there are 3 design approaches:
I am trying to do 2. at this time, though 3. is kinda the same work in reality, depepending on the philosophy of being more generic or more concise in the data model. 3 is more generic and will make it easier to add more features (without migration impact) |
Have a PoC written, going through unit tests ATM |
Hi, sorry for delay in responding, great that you were willing to take on this task force 😄 assigned. I like the approach which You proposed and which You are currently working on (the one with holding data about deleted topics in zk in "deletion info") |
see allegro#1739 Deletion creates a node in the group /deletion_time, stores the time there Throws an exception if not enough time has passed (5 min) all existing unit tests pass (except the docker) pending: - updating with latest upstream - manual testing in UI and running docker test - adding unit tests for the added functionality - checking integration tests - configuration of the time delay
Feel free to wait until TODOs are completed but early feedback welcome. |
See PR open for review, it'd be awesome if the hacktoberfest accepted tag can be added :) |
I will check the remaining TODOs tomorrow |
Currently hermes-management has a bug which is caused by deleting topic and then recreating it quickly
The story is similar every time:
Issue was thought to be solved with upgrade to kafka client 2.8.2 but again appeared recently. We would like to have a workaround for this.
One of the proposed solutions is to introduce "grace period" for deleted topics. E.g. if someone deletes a topic we should block the creation of topic with same name for long enough so that cluster and kafka producers can be in consistent state. Probably > 5 minutes is enough because metadata is refreshed every 5 minutes.
The text was updated successfully, but these errors were encountered: