beacon-fuzz

Open-source differential fuzzing of Ethereum 2.0 Phase 0 implementations. Maintained by Sigma Prime for the Ethereum Foundation.

Overview

A differential fuzzer of Eth2.0 implementations using libfuzzer. By default, fuzzing progresses indefinitely unless an implementation panics or differing output is identified.

This is a continuation of Guido Vranken's eth2.0-fuzzing.

This project and its inner workings are subject to change.

A note on terminology: "client" and "implementation" are used interchangeably here to mean a specific Eth2 implementation.

This README focuses on our differential fuzzing effort, please refer to eth2fuzz for panic/crashes detection.

Community Fuzzing

For details on our community fuzzing initative, please refer to the eth2fuzz README, along with this blog post.

Current Status

Currently fuzzes against Eth2 v0.10.1 python or Go executable specs (pyspec or zrnt)

Implementations

• Lighthouse (rust)
• Nimbus (nim)
• pyspec (python)
• Trinity (python)
• zrnt (go)

Operational Fuzz Targets:

(and their relevant spec function)

All currently use the "mainnet" config: https://github.com/ethereum/eth2.0-specs/blob/v0.10.1/configs/mainnet.yaml

• attestation - process_attestation
• attester_slashing - process_attester_slashing
• block - state_transition
• block_header - process_block_header
• deposit - process_deposit
• proposer_slashing - process_proposer_slashing
• shuffle - compute_shuffled_index
• voluntary_exit - process_voluntary_exit

See corpora repository for explanation of input structure.

Corpora

See corpora for examples and explanation of structure.

Usage

Quickstart:

$ git clone --depth 1 --recurse-submodules https://github.com/sigp/beacon-fuzz.git
$ git clone --depth 1 https://github.com/sigp/beacon-fuzz-corpora.git
$ cd beacon-fuzz
$ sudo ./runfuzzer.sh block_header ../beacon-fuzz-corpora/0-9-1/mainnet/block_header/ ../beacon-fuzz-corpora/0-9-1/mainnet/beaconstate

Interactive usage:

$ git clone --depth 1 --recurse-submodules https://github.com/sigp/beacon-fuzz.git
$ cd beacon-fuzz
$ sudo docker build . -t beacon_fuzz
$ sudo docker run -it beacon_fuzz bash
# git clone --depth 1 https://github.com/sigp/beacon-fuzz-corpora.git
# export ETH2_FUZZER_STATE_CORPUS_PATH="/eth2/beacon-fuzz-corpora/0-9-1/mainnet/beaconstate"
# /eth2/fuzzers/attestation/fuzzer /eth2/beacon-fuzz-corpora/0-9-1/mainnet/attestation

Use jobs=N to run N simultaneous jobs (by default in num_cpu_cores/2 processes).

Use help=1 for more arguments (see also libfuzzer docs)

Roadmap

• Improved onboarding, ease of adding new targets and implementations
• Improved coverage measurements and visibility
• Deploy on dedicated production fuzzing infrastructure
• Structure-aware fuzzing mutations

Contributing

Use pre-commit

$ pre-commit install

(see also .pre-commit-config.yaml)

If build fails, comment the RUN /eth2/build.sh in Dockerfile, and run it manually from within the container. Can adjust Makefiles as needed.

It is generally fine to run build.sh multiple times, and previously built components will be ignored.

makeing fuzzers directly

This is quicker than re-running ./build.sh and is useful when troubleshooting specific build issues.

After running build.sh once, a file /eth2/exported_env.sh will be created. Sourcing this will ensure you have all the environment variables required by the Makefiles.

NOTE: the fuzzer's Makefiles do not currently identify changes to dependent file outside of the fuzzers directory. So, for example, a change to a dependent file in ./files/lib will require a make clean (or equivalent) for the modifications to be visible.

Compile-time options

The following make variable flags are exposed (and are enabled by setting them to anything other than ''): e.g. make BFUZZ_NO_DISABLE_BLS=1 all

• BFUZZ_NO_DISABLE_BLS - don't disable bls verification

Adding new implementations for a target

See pyspec harness.pys for a succinct, readable harness implementation example without much boilerplate.

For state transition functions, each client should expect to receive a correctly-encoded SSZ container containing a BeaconState, and an input object relevant for the transition. (As described in https://github.com/sigp/beacon-fuzz-corpora/blob/master/0-8-3/README.md, except state_id has been replaced with a corresponding BeaconState.)

Please panic/abort if SSZ decoding fails, as this indicates an error in preprocessing or the SSZ libraries. e.g. even though a client implementing the Attestation fuzz target can expect to receive any arbitrary Attestation object, it should be in the form of a validly-encoded SSZ blob.

Currently clients will only receive known, valid BeaconStates (from ETH2_FUZZER_STATE_CORPUS_PATH) so the actual fuzzing/mutation is performed with the other part of the input. This is because clients generally maintain their own BeaconStates, so don't expect to receive arbitrary states from untrusted sources. (It is also highly unlikely that current mutation will ever produce a valid BeaconState)

There are 3 types of results/outputs that a client is expected to return to the fuzzer:

1. A bytestring/bytearray/blob (usually a SSZ-encoding of the BeaconState post-transition).
2. An error result (usually a nullptr, None or False equivalent).

• To be returned when the operation failed but the client is in a consistent state (e.g. supplied Attestation data does not refer to an appropriate epoch).
• The c++ module returns this as a std::nullopt.
• This is necessary to differentiate from the few cases where an empty bytestring is a valid and successful result e.g. shuffling an empty list.

3. Abort/panic.

• To occur when a client is in an inconsistent state and indicates a bug is present.

Known bugs/limitations

• Python editable installs in Venvs aren't detected.

Trophies

The fuzzing tools developed as part of this project (eth2fuzz, eth2diff and beacon) helps to find the following bugs inside eth2 clients.

Nimbus

• Nimbus: process_attestation missing index validation fixed
• Nimbus: process_deposit not validating merkle proofs fixed
• Nimbus: ncli_pretty Deposit SSZ parsing AssertionError fixed
• Nimbus: ncli_pretty Bytes Reader IndexError decoding BeaconState with empty container fixed
• Nimbus: ncli_pretty Bytes Reader IndexError decoding BeaconState with variable list reporting 0 length fixed
• Nimbus: ncli_transition out of memory segfault during process_final_updates fixed
• Nimbus: ncli_transition AssertionError due to inconsistent aggregation bits and committee length when passed invalid BeaconState and BeaconBlock (See 1) fixed
• Nimbus: ncli IndexError decoding 0-byte SSZ BitList fixed
• Nimbus: IndexError during AttesterSlashing processing fixed

Trinity

• Trinity: some block validation raising IndexError, not caught by BeaconChainSyncer

Teku

• Teku: infinite loop when decoding SSZ BitList without "end-of-list" marker bit fixed
• Teku: transition subcommand raising IllegalArgumentException instead of logging when passed invalid SSZ fixed
• Teku: transition subcommand raising IllegalArgumentException instead of logging when passed invalid SSZ fixed
• Teku: IndexOutOfBoundsException when SSZ decoding 0-byte BitList fixed
• Teku: IndexOutOfBoundsException when passed invalid BeaconState and committee size is inconsistent with attestation aggregation bits. (See 1) fixed
• Teku: ArrayIndexOutOfBoundsException in AttesterSlashing processing

Lighthouse

• Lighthouse: out-of-bounds offset in variable list SSZ decoding fixed
• Lighthouse: multiplication overflow in compute_proposer_index (See 1) fixed
• Lighthouse: ENR panic fixed
• Lighthouse: Underflow in Snappy (external dependency) fixed

Lodestar

• Lodestar: TypeError when SSZ decoding a Block with invalid BigInt parent scope fixed
• Lodestar: RangeError when SSZ decoding an empty Block container
• Lodestar: TypeError when decoding invalid ENR string fixed
• Lodestar: TypeError: public key must be a Buffer when decoding invalid ENR string fixed
• Lodestar: memory exhaustion / OOM when parsing invalid ENR string

Prysm

• Prysm: panic: runtime error: slice bounds out of range when parsing SSZ container fixed
• Prysm: panic: runtime error: nil pointer dereference when processing ProposerSlashing fixed

1: NOTE BeaconState objects are considered trusted inputs (for the moment), so client state transition functions are not expected to handle invalid BeaconState values, for now.

License

MIT - see LICENSE