-
Notifications
You must be signed in to change notification settings - Fork 172
Improve: Ensure uniform membership config during config changes #1351
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Improve: Ensure uniform membership config during config changes #1351
Conversation
That's a nice, minimal way to implement this flattening operation :) Now we just need to think of a way to make the |
616301b
to
27dbd28
Compare
This commit improves the handling of membership configuration changes to ensure that the system reliably transitions to a uniform configuration in the second step of a joint membership change. ### Problem: During a membership config change, there are two steps: 1. Transition to a joint configuration containing both `Cold` (current config) and `Cnew` (new config). 2. Transition to a uniform configuration containing only `Cnew`. Previously, the second step attempted to apply the same change on the current membership state. If multiple membership change requests were processed in parallel, this approach could result in the system being left in a joint configuration. For example: - Initial config: `{a, b, c}`. - Task 1: `AddVoterIds(x)`. After the first step: `[{a, b, c}, {a, b, c, x}]`. - Task 2: `RemoveVoters(x)`. After the first step: `[{a, b, c, x}, {a, b, c}]` (applied on the last config `{a, b, c, x}`). - Task 1 proceeds to the second step, re-applies `AddVoterIds(x)`, and the result is `[{a, b, c}, {a, b, c, x}]`. - The system remains in a joint configuration, which is unintuitive and contradicts standard Raft expectations. ### Solution: The second step now applies an **empty change request** (`AddVoterIds({})`) to the last configuration in the current joint config. This ensures that the system always transitions to a uniform configuration in the second step. ### Impact: - No behavior changes occur if only one membership change request is in progress. - If multiple requests are processed concurrently, the application must still verify the result, and the new behavior ensures the system transitions to a uniform state. This fix prevents the system from being left in an unintended joint configuration, improving consistency and adherence to Raft principles. Thanks to @tvsfx for providing feedback on this issue and offering a detailed explanation of the solution.
27dbd28
to
1e5aee2
Compare
Oh, I just had a thought: I don't think this implementation works for removing |
Hmm... correct |
@tvsfx |
Changelog
Improve: Ensure uniform membership config during config changes
This commit improves the handling of membership configuration changes to
ensure that the system reliably transitions to a uniform configuration
in the second step of a joint membership change.
Problem:
During a membership config change, there are two steps:
Cold
(currentconfig) and
Cnew
(new config).Cnew
.Previously, the second step attempted to apply the same change on the
current membership state. If multiple membership change requests were
processed in parallel, this approach could result in the system being
left in a joint configuration. For example:
Initial config:
{a, b, c}
.Task 1:
AddVoterIds(x)
. After the first step:[{a, b, c}, {a, b, c, x}]
.Task 2:
RemoveVoters(x)
. After the first step:[{a, b, c, x}, {a, b, c}]
(applied on the last config{a, b, c, x}
).Task 1 proceeds to the second step, re-applies
AddVoterIds(x)
, andthe result is
[{a, b, c}, {a, b, c, x}]
.The system remains in a joint configuration, which is unintuitive and
contradicts standard Raft expectations.
Solution:
The second step now applies an empty change request
(
AddVoterIds({})
) to the last configuration in the current jointconfig. This ensures that the system always transitions to a uniform
configuration in the second step.
Impact:
No behavior changes occur if only one membership change request is in
progress.
If multiple requests are processed concurrently, the application must
still verify the result, and the new behavior ensures the system
transitions to a uniform state.
This fix prevents the system from being left in an unintended joint
configuration, improving consistency and adherence to Raft principles.
Thanks to @tvsfx for providing feedback on this issue and offering a
detailed explanation of the solution.
This change is