Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify and improve key generation when working in cluster mode. #206

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

ushachar
Copy link
Contributor

@ushachar ushachar commented Feb 7, 2023

Instead of randomly generating keys and storing them per shard until we get a key which matches our desired shard, generate one which will always match.
This improved performance and reduces memory overhead when working with a lot of primaries.

@ushachar ushachar force-pushed the uri_cluster_key_generation branch 2 times, most recently from 30b1e91 to f8019be Compare February 7, 2023 22:17
@filipecosta90
Copy link
Collaborator

@ushachar / @YaacovHazan can we merge #203 before this one to ensure all code added here is covered on the flow tests. Assuming it is but worth the double check. agree?

@ushachar
Copy link
Contributor Author

ushachar commented Feb 8, 2023

@ushachar / @YaacovHazan can we merge #203 before this one to ensure all code added here is covered on the flow tests. Assuming it is but worth the double check. agree?

Sure -- +1'ed #203
This PR doesn't add any tests - but should slightly improve coverage since it removed a big chunk of code.
note that I am working on a test around the handling of MOVED.

Instead of randomly generating keys and storing them per shard until
we get a key which matches our desired shard, generate one which will
always match.
This improved performance and reduces memory overhead when working with
a lot of primaries.
@ushachar
Copy link
Contributor Author

ushachar commented Feb 8, 2023

@filipecosta90 rebased to include coverage

@codecov-commenter
Copy link

Codecov Report

Merging #206 (633321e) into master (78b0a9a) will decrease coverage by 0.34%.
The diff coverage is 100.00%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##           master     #206      +/-   ##
==========================================
- Coverage   55.90%   55.56%   -0.34%     
==========================================
  Files          21       21              
  Lines        4270     4233      -37     
==========================================
- Hits         2387     2352      -35     
+ Misses       1883     1881       -2     
Impacted Files Coverage Δ
cluster_client.cpp 66.66% <100.00%> (-3.19%) ⬇️
shard_connection.h 87.50% <0.00%> (-12.50%) ⬇️
protocol.cpp 36.23% <0.00%> (-0.32%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@filipecosta90 filipecosta90 self-requested a review February 8, 2023 13:56
if (m_config->requests > 0 && m_reqs_generated >= m_config->requests)
return false;
}
*key_index = m_obj_gen->get_key_index(iter);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ushachar in cluster mode we're growing the keyspace range to above the max/min range. This is because we're using the crc16 + key index, meaning we can now have #primaries * keyspace range. The PR #208 includes tests to track this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants