Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

branchmark storage: Arg to read a fraction of all KV pairs #146

Closed
ggwpez opened this issue Jul 27, 2023 · 6 comments
Closed

branchmark storage: Arg to read a fraction of all KV pairs #146

ggwpez opened this issue Jul 27, 2023 · 6 comments
Assignees
@adelarja
Labels
C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. D0-easy Can be fixed primarily by duplicating and adapting code by an intermediate coder. T1-FRAME This PR/Issue is related to core FRAME, the framework.

Comments

@ggwpez
Copy link
Member

ggwpez commented Jul 27, 2023 •
edited
Loading

👉 Good issue for newcomers (please dont pick up when you already have experience).

The Moonbeam storage benchmark takes about 40 hrs.
I order to get some faster preliminary results we should add a --access-fraction % argument to only access a percentage of all KV pairs.

Maybe wen can still think of a nicer name than --access-fraction before implementing it.
@ggwpez ggwpez mentioned this issue Jul 27, 2023
Closed
@Doordashcon
Copy link
Contributor

Hello @ggwpez, currently looking into this

@ggwpez
Copy link
Member Author

ggwpez commented Aug 1, 2023

Thanks @Doordashcon, since it is extremely easy, i would rather like to keep this for a newcomer that has not yet worked with Substrate...

@Doordashcon
Copy link
Contributor

Hello @ggwpez, Makes sense

@ggwpez ggwpez mentioned this issue Aug 21, 2023
Open
@adelarja
Copy link
Contributor

Hi @ggwpez, I can work on this ticket if there's no problem. I've read through this ticket to comprehend the issue.

Are there any related issues or resources that could help me with this?

@ggwpez
Copy link
Member Author

ggwpez commented Aug 22, 2023

Are there any related issues or resources that could help me with this?

Hm, not sure. Mostly just reading the code I guess. Entry point for adding the argument would be here https://github.com/paritytech/substrate/blob/96f8c97dff7f419a349ee01d02613445ba4a41f8/utils/frame/benchmarking-cli/src/storage/cmd.rs#L75

@juangirini juangirini transferred this issue from paritytech/substrate Aug 24, 2023
@the-right-joyce the-right-joyce added C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. T1-FRAME This PR/Issue is related to core FRAME, the framework. D0-easy Can be fixed primarily by duplicating and adapting code by an intermediate coder. and removed Z6-mentor labels Aug 25, 2023
@adelarja adelarja mentioned this issue Aug 29, 2023
Closed
@ggwpez
Copy link
Member Author

ggwpez commented Feb 5, 2024

Turns out this does not work since the DB backend cannot give us a random KV pair.

@ggwpez ggwpez closed this as completed Feb 5, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Runtime / FRAME Feb 5, 2024
@github-project-automation github-project-automation bot moved this from In progress to Done in Benchmarking and Weights Feb 5, 2024
lexnv added a commit that referenced this issue Apr 3, 2024
@lexnv @jsdw
transactionWatch: Limit the number of active subscriptions and clarif…
bef3ab5 
…y error codes (#146)

* txWatch: Limit the number of active subscriptions

Signed-off-by: Alexandru Vasile <[email protected]>

* txWatch: Specify rpc error code from jsonrpc.org

Signed-off-by: Alexandru Vasile <[email protected]>

* txWatch: Add mention for invalid tx decoded bytes

Signed-off-by: Alexandru Vasile <[email protected]>

* Update src/api/transactionWatch_unstable_submitAndWatch.md

Co-authored-by: James Wilson <[email protected]>

---------

Signed-off-by: Alexandru Vasile <[email protected]>
Co-authored-by: James Wilson <[email protected]>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
457c5d4 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 8, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
22df1c5 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
d52cce0 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
350905f 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
595d1fd 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
0c1b138 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
cddd453 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 9, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
3ef3f13 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
53e1784 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
serban300 pushed a commit to serban300/polkadot-sdk that referenced this issue Apr 10, 2024
@dependabot-preview @serban300
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
7f55e1a 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
jonathanudd pushed a commit to jonathanudd/polkadot-sdk that referenced this issue Apr 10, 2024
@dependabot-preview @bkchr
Bump derive_more from 0.99.7 to 0.99.8 (paritytech#146)
800c431 
Bumps [derive_more](https://github.com/JelteF/derive_more) from 0.99.7 to 0.99.8.
- [Release notes](https://github.com/JelteF/derive_more/releases)
- [Changelog](https://github.com/JelteF/derive_more/blob/master/CHANGELOG.md)
- [Commits](JelteF/derive_more@v0.99.7...v0.99.8)

Signed-off-by: dependabot-preview[bot] <[email protected]>

Co-authored-by: dependabot-preview[bot] <27856297+dependabot-preview[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adelarja adelarja

Labels
C1-mentor A task where a mentor is available. Please indicate in the issue who the mentor could be. D0-easy Can be fixed primarily by duplicating and adapting code by an intermediate coder. T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
Benchmarking and Weights
Status: Done
Runtime / FRAME
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Benchmark storage arg to read fraction of kv pairs. adelarja/polkadot-sdk
5 participants
@juangirini @ggwpez @adelarja @Doordashcon @the-right-joyce