Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unnecessary merge attempts #18

Open
kondziu opened this issue Jul 16, 2021 · 0 comments
Open

Unnecessary merge attempts #18

kondziu opened this issue Jul 16, 2021 · 0 comments
Labels
enhancement New feature or request good first issue Good for newcomers v4 wishlist Things to do for v4.

Comments

@kondziu
Copy link
Contributor

kondziu commented Jul 16, 2021

Whenever multiple subsets of a dataset are used, a merge happens. The merged data is cached, so often the merge, finds nothing to do in the end, but it still takes time to get through the data. Find a way to skip merge attempts when we know a priori that nothing has changed. One idea is to store some sort of checksum-like metric that can be quickly calculated for the pre-merged subsets and checked before attempting to merge, on the theory, that if these are correct, the currently cached merged dataset can be used and the process of merging is skipped entirely.

Related to #6.

@kondziu kondziu added enhancement New feature or request good first issue Good for newcomers labels Jul 16, 2021
@kondziu kondziu added the v4 wishlist Things to do for v4. label Jul 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers v4 wishlist Things to do for v4.
Projects
None yet
Development

No branches or pull requests

1 participant