Description
Expected Behavior
When the sequence zookeeper node counter overflows (as documented in: https://zookeeper.apache.org/doc/r3.6.0/zookeeperProgrammers.html#Sequence+Nodes+--+Unique+Naming), it would be expected that there is one re-election from 2147483647 to -2147483648. Afterwards, incrementing in the negative space should be stable again and not causing unexpected re-elections.
Actual Behavior
When overflowing and reaching the negative space starting from -2147483648, we observed perpetual leader re-elections. With each new added sequence number, the new instance would register itself as the new leader.
When digging into the kazoo code, it seems to be a typing issue, as the sequence is matched using a regex but not cast to an integer, meaning checking for the smallest sequence number is a String comparison, i.e. "-2147483648" > "-2147483647"
is True
.
Snippet to Reproduce the Problem
N/A
Logs with logging in DEBUG mode
N/A
Specifications
- Kazoo version: 2.9
- Zookeeper version: 3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f
- Python version: 3.11
- OS: SLES 12 SP2