Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add cli flags for storage capacity per subnetwork #1558

Merged
merged 2 commits into from
Oct 29, 2024

Conversation

morph-dev
Copy link
Collaborator

@morph-dev morph-dev commented Oct 27, 2024

What was wrong?

Based on #1548 and discussion in the last meeting, we agreed that we want following cli flags regarding storage capacity:

  1. storage.total flag that specifies total storage capacity (in MB)
    • this capacity should be shared between all enabled networks
    • current flag mb should be set as alias (until we migrate in other places)
  2. storage.beacon, storage.history and storage.state that specifies storage capacity (in MB) per subnetwork explicitly

Some restrictions:

  • storage.total can't be combined with other flags
    • other flags can/should be used together, see next point
  • if storage.{subnetwork} is specified for one enabled subnetwork, it should be specified for all enabled networks

For future work:

  • if user didn't enable subnetwork, but we detect that it's using some storage on disk (e.g. from previous run), print warning message (or do something else, TBD).

How was it fixed?

  • created cli flags
    • added detailed documentation on how they should be used
  • added checks when TrinConfig is created
    • added tests
  • adapted logic for creating PortalStorageConfig
    • added tests

To-Do

@morph-dev morph-dev self-assigned this Oct 27, 2024
@morph-dev morph-dev marked this pull request as ready for review October 27, 2024 12:43
Copy link
Member

@KolbyML KolbyML left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@morph-dev over all the PR looks good

Copy link
Collaborator

@carver carver left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love all the tests!

nit I just did a quick grep, and didn't see any tests that the --mb alias still works. I'd be in favor of a single simple one to make sure we don't accidentally disable it in the future.

trin-storage/src/config.rs Show resolved Hide resolved
Comment on lines +398 to +406
if storage_flag.is_some() && !config.portal_subnetworks.contains(&subnetwork) {
return Err(Error::raw(
ErrorKind::ArgumentConflict,
format!(
"--storage.{} is set but {subnetwork} subnetwork is not enabled",
subnetwork.to_cli_arg(),
),
));
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we have to error out in this case. I do like the impulse to be strict to avoid accidents.

But I can also imagine a scenario where I'm flipping between enabling and disabling a network, and having to do that in two places in the command being a little annoying.

I'm slightly in favor of dropping this if you're neutral.

@@ -256,7 +301,7 @@ impl TrinConfig {

pub fn new_from<I, T>(args: I) -> Result<Self, clap::Error>
where
I: Iterator<Item = T>,
I: IntoIterator<Item = T>,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do really like this change. It also added noise to a big PR. I think it belongs in its own commit/PR.

assert_eq!(
config.storage_capacity_config(),
StorageCapacityConfig::Specific {
beacon_mb: Some(0),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think there are any actual bugs here, but as a practice the test for setting a specific value should probably pick one that isn't the value you would arrive at by default.

@morph-dev
Copy link
Collaborator Author

Rebased, fixed default capacity tests, added --mb and zero capacity tests, and reverted partially reverted refactoring changes.
I will merge it like this.

Let me know if you have other comments and I will address them in the followup PR.

@morph-dev morph-dev merged commit 741d656 into ethereum:master Oct 29, 2024
9 checks passed
@morph-dev morph-dev deleted the cli branch October 29, 2024 11:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants