Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Stop replication is failing when ultrawarm migration put index.blocks.ultrawarm_allow_delete setting on follower index #1337

Open
nisgoel-amazon opened this issue Feb 27, 2024 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@nisgoel-amazon
Copy link
Contributor

What is the bug?
There is an issue which can happen when stop replication is delayed for the follower index due to automated snapshot in progress.

There is an ISM policy defined in leader domain to rollover the index and do ultrawarm migration.

When index in leader domain is rollover and ultrawarm migration starts, follower domain is notified and it tries to stops the replication. But when automated snapshot is in progress, SnapshotInProgressException is coming while stoping the replication.

Meanwhile leader index moves to ultrawarm node, and applies following settings to the leader index.
{"index.auto_expand_replicas":"false","index.blocks.ultrawarm_allow_delete":"true","index.refresh_interval":"-1"}

As replication is still running the setting are applied to the follower index, and later when snapshot is successful, Index moves to close state to stop replication but as settings were applied replication fails, and index remains in close state.

What is the expected behavior?
Stop replication call should be successful after snapshot is successful. And Index should not left in closed state.

What is your host/environment?

  • OS: 2.5
  • CCR Plugins
@nisgoel-amazon nisgoel-amazon added bug Something isn't working untriaged labels Feb 27, 2024
@mohitamg mohitamg self-assigned this Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants