Batch sstables basing on percentage of the node's data #4059
Comments
Michal-Leszczynski
added a commit
that referenced
this issue
Oct 3, 2024
This commit allows to set --batch-size=0. When this happens, batches will be created so that they contain about 5% of expected node workload during restore. This allows for creating big, yet evenly distributed batches without the need to play with the --batch-size flag. It should also work better fine when backed up cluster had different amount of nodes than the restore destination cluster. Fixes #4059
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As described in this comment, SM should make it easier to set optimal --batch-size for restoring as fast as possible.
In order to achieve that, we should add a special
--batch-size=0value meaning that we want to create batches that result in the fastest restore. This follows the convention used by other flags like--parallelor--rate-limit.Setting
--batch-size=0would result in sending batches of unlimited amount of sstables, but of size up to 5% of expected node workload.The text was updated successfully, but these errors were encountered: