Add EIP: Hardware and Bandwidth Recommendations for Validators and Full Nodes #9270
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,145 @@ | ||
| --- | ||
| eip: 7870 | ||
| title: Hardware and Bandwidth Recommendations for Validators and Full Nodes | ||
|
Check failure on line 3 in EIPS/eip-7870.md
|
||
| description: System recommendations for Validators and Full nodes | ||
| author: Parithosh Jayanthi (@parithosh), Kevaundray Wedderburn (@kevaundray), Josh Rudolf (@jrudolf), Dankrad Feist (@dankrad), Justin Traglia (@jtraglia), Ignacio Hagopian (@jsign), George Kadianakis (@asn-d6), Fredrik Svantes (@fredriksvantes) | ||
| discussions-to: https://ethereum-magicians.org/t/hardware-and-bandwidth-recommendations-for-full-nodes-and-validators/22675 | ||
| status: Draft | ||
| type: Informational | ||
| created: 2025-01-26 | ||
| --- | ||
|
|
||
| ## Abstract | ||
|
|
||
| This proposal specifies hardware and bandwidth recommendations for different types of Ethereum nodes: | ||
|
|
||
| - **Full nodes**: Nodes that follow the tip of the chain without necessarily proposing blocks. | ||
| - **Validators**: Split into: | ||
| - **Attesters**: Validators that validate and attest to blocks created by proposers. | ||
| - **Local block builders** (Proposers): Validators that create blocks locally and broadcast them to the network. | ||
|
|
||
| The resource-intensive aspect for local block builders lies in creating the block and quickly broadcasting the data required for attesters to validate it in time. | ||
|
|
||
| We note that it may be possible to run a client with less than the recommended specifications, however benchmarks and decision-making will be made with respect to these recommendations. | ||
|
|
||
| ## Motivation | ||
|
|
||
| Clear system specifications are crucial for: | ||
|
|
||
| - Ensuring meaningful benchmark comparisons across different client implementations. | ||
| - Enabling informed decision-making about protocol upgrades and their resource usage implications. | ||
| - Providing clear guidance for node operators to ensure alignment with future network requirements. | ||
|
|
||
| Without a shared understanding of target hardware specifications: | ||
|
|
||
| - Benchmark results lose significance due to inconsistent testing environments. | ||
| - Decision-making becomes challenging for implementation choices, as performance characteristics are heavily hardware-dependent. | ||
| - Network participants lack clear guidance for hardware investments. | ||
|
|
||
| ## Specification | ||
|
|
||
| ### Roles and Their Recommended Specifications | ||
|
|
||
| Node operators typically run both an **Execution Layer (EL)** client and a **Consensus Layer (CL)** client on the same machine. The specifications below assume the combined resource usage of both. | ||
|
|
||
| | Node Type | Storage | Memory | CPU Cores / Threads | **PassMark CPU Rating** | Bandwidth Download / Upload | | ||
| | ----------------------- | --------- | ------ | ------------------- | ----------------------- | --------------------------- | | ||
| | **Full Node** | 4 TB NVMe | 32 GB | 4c / 8t | ~1000 ST / 3000 MT | 50 Mbps / 15 Mbps | | ||
| | **Attester** | 4 TB NVMe | 64 GB | 8c / 16t | ~3500 ST / 25000 MT | 50 Mbps / 25 Mbps | | ||
| | **Local Block Builder** | 4 TB NVMe | 64 GB | 8c / 16t | ~3500 ST / 25000 MT | 100 Mbps / 50 Mbps | | ||
|
|
||
| *Approximate single-thread (ST) and multi-thread (MT) PassMark CPU scores. For example, a PassMark ST rating of 3500 and an MT rating of 25000 typically corresponds to upper mid-range server CPUs circa 2024–2025.* | ||
|
|
||
| ## Rationale | ||
|
|
||
| ### Storage | ||
|
|
||
| - **Recommended**: 4 TB NVMe M.2 drive with: | ||
| - **Sequential R/W**: 7,000 MB/s | ||
| - **Random 4K R/W**: Up to 1,000,000 IOPS | ||
| - **Why NVMe over SATA?** | ||
| - NVMe drives have significantly higher throughput and lower latency than SATA SSDs. | ||
| - Drives without DRAM (DRAMless) or with QLC flash are not advised, due to lower endurance and potentially lower sustained performance. | ||
| - **On Endurance (TBW)** | ||
| - Running a node involves frequent writes (e.g., database updates, logs). Ensure that the SSD’s Total Bytes Written (TBW) rating is sufficient for multi-year operation. | ||
| - **Capacity Considerations** | ||
| - As of January 2025, 2 TB can still work, but daily chain growth makes 4 TB more future-proof. | ||
| - While EIP-4444 aims to reduce historical storage requirements, the timeline for EIP-4444 remains uncertain. | ||
|
|
||
| ### Memory | ||
|
|
||
| - **Why 64 GB for Validators (Attester / Local Block Builder)?** | ||
| - Running an Ethereum validator (both EL & CL clients) can be memory-intensive, with state cache dominating RAM usage. | ||
| - Certain memory intensive client combinations have historically failed with 16 GB. | ||
| - Preliminary benchmarks (e.g., [zk-STARK memory usage](https://hackmd.io/@han/bench-hash-in-snark)) suggest cryptographic operations can demand significant RAM. | ||
|
Check failure on line 74 in EIPS/eip-7870.md
|
||
| - Relative to total hardware costs, upgrading from 32 GB to 64 GB is a small price but can improve stability and is more future proof with regards to zk-STARKS. | ||
|
|
||
| ### CPU | ||
|
|
||
| - **Single vs. Multi-thread Performance** | ||
| - Attesters and local block builders benefit from both high single-thread and multi-thread performance. | ||
|
|
||
| ### Bandwidth | ||
|
|
||
| #### Local Block Builders | ||
|
|
||
| - **Recommended**: 100 Mbps download, 50 Mbps upload. | ||
| - **Rationale**: | ||
| - Distributing blocks is highly bandwidth-intensive. | ||
| - If a builder cannot meet these speeds, they risk slower propagation and causing late blocks. | ||
| - In extreme cases, a low-bandwidth node could propose partially full blocks or one that includes fewer blobs to mitigate slow broadcast. | ||
|
|
||
| #### Attesters | ||
|
|
||
| - **Recommended**: 50 Mbps download, 25 Mbps upload. | ||
| - **Minimum tested**: 15 Mbps (as per [ethPandaOps](https://ethpandaops.io/) simulations where 40% of the network had Gigabit connections and the other 60% had 15 Mbps upload). | ||
|
Check failure on line 95 in EIPS/eip-7870.md
|
||
| - However, real-world performance depends on peer network topology. A node with poor bandwidth could in theory quickly share data with a well-connected peer with good bandwidth which means that this peer could quickly seed the network. | ||
| - **Rationale**: | ||
| - 25 Mbps was chosen as a buffer to account for these miscellaneous factors that are harder to predict. | ||
|
|
||
| #### Validators Using MEV-Boost | ||
|
|
||
| - **Recommended**: 50 Mbps download / 25 Mbps upload. | ||
| - **Rationale**: | ||
| - Most MEV-relay interactions involve fetching bundles and block headers, which can be done within typical broadband speeds. | ||
| - While the local validator will also share the block with its peers, the relay will do the same which reduces the need for local bandwidth. | ||
| - However, there may be cases where your validator will still build a local block, such as if no MEV-relay responds or if the value of the MEV reward is lower than the minimum configuration set in MEV-Boost. In these circumstances, the recommendations for **Local Block Builders** would be relevant. | ||
|
|
||
| #### Full Nodes | ||
|
|
||
| - **Recommended**: 50 Mbps download / 15 Mbps upload. | ||
| - **Rationale**: | ||
| - Full nodes currently participate in sampling and must track the chain tip, but are not as latency-sensitive as attesters or local block builders. | ||
| - They can operate on lower bandwidth but risk being a slot or more behind the chain if bandwidth capacity is severely limited. | ||
|
|
||
| ## Quick Reference Summary | ||
|
Check failure on line 115 in EIPS/eip-7870.md
|
||
|
|
||
| - **Full Node** | ||
| - **Storage**: 4 TB NVMe | ||
| - **RAM**: 32 GB | ||
| - **CPU**: 4 cores / 8 threads (~1000 ST / ~3000 MT PassMark) | ||
| - **Bandwidth**: 50 Mbps down / 15 Mbps up | ||
|
|
||
| - **Attester** | ||
| - **Storage**: 4 TB NVMe | ||
| - **RAM**: 64 GB | ||
| - **CPU**: 8 cores / 16 threads (~3500 ST / ~25000 MT PassMark) | ||
| - **Bandwidth**: 50 Mbps down / 25 Mbps up | ||
|
|
||
| - **Local Block Builder** | ||
| - **Storage**: 4 TB NVMe | ||
| - **RAM**: 64 GB | ||
| - **CPU**: 8 cores / 16 threads (~3500 ST / ~25000 MT PassMark) | ||
| - **Bandwidth**: 100 Mbps down / 50 Mbps up | ||
|
|
||
| ## Backwards Compatibility | ||
|
|
||
| This EIP is informational and requires no protocol changes. We recommend that future EIPs include an assessment of their impact on these hardware recommendations. | ||
|
|
||
| ## Security Considerations | ||
|
There was a problem hiding this comment. I think I'd like to see a section about "Future". What could we expect in the future? Would we need more storage? If I build a node using the recommended settings, what is the target duration that this setup is usable and needs no hardware upgrades? There was a problem hiding this comment. Off the top of my head, this is what I can think of. Kind of hard to be comprehensive and I'm sure there are nuances that are missed or mistakes: Full node
Attesters
Block builder
(One note here is that until we have zkvm proofs, you can still block build but at a lower throughput by setting the maximum number of blobs you can do in your EL based on your bandwidth requirements) I think the main entity that will change significantly is the block builder. Other changes to consider:
The argument here is that if we increase the gas limit, we increase the rate of state growth and eventually no-one but centralized parties will be able to hold the state and they could deny access or censor the rest of the network. But with state network, we would always be able to retrieve the state (at a slower throughput).
It has been argued that this might give folks reason to pump up the requirements for the centralizing components even further like block building. I am unsure about how this will play out. There was a problem hiding this comment. One thing I'd suggest doing prior to applying that is to do any research among community either by pool on X or anything else which can have bigger outreach. If it will appear that 51% of users have weaker than specified setup (which is super unlikely but still) we could potentially open a route for some unexpected attacks because we will test on too good hardware. At least worth to know in case of noticed issue which affect weaker setups how much network (in terms of nodes) is put in danger. |
||
|
|
||
| N/A | ||
|
|
||
| ## Copyright | ||
|
|
||
| Copyright and related rights waived via [CC0](../LICENSE.md). | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are all those numbers applied based on some research made? Or based on experience from discussions with stakers?
From Nethermind perspective those are super comfortable.
Also shall we distinguish EL and CL requirements to be treated separately? I saw a lot of setups where actually both are running on different machines.