Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ProjectTracking]: Epoch Sync V4 #73

Open
20 of 28 tasks
Tracked by #82
robin-near opened this issue Aug 13, 2024 · 0 comments
Open
20 of 28 tasks
Tracked by #82

[ProjectTracking]: Epoch Sync V4 #73

robin-near opened this issue Aug 13, 2024 · 0 comments
Assignees
@robin-near
Labels
project-tracking

Comments

@robin-near
Copy link

robin-near commented Aug 13, 2024 •
edited
Loading

Overview

Issues (ongoing list):

Critical tasks:

Tasks
Beta Give feedback
  1. [Epoch Sync] Implement EpochSyncProof validation nearcore#11932
    staffik
  2. [Epoch Sync] Network message size optimization nearcore#11933
  3. [Epoch Sync] Implement deriving EpochSyncProof from an older EpochSyncProof nearcore#11935
  4. [Epoch Sync] Implement network rate limiting and caching for EpochSyncRequest nearcore#11937
  5. [Epoch Sync] Blocks received during state sync marked as invalid nearcore#11955
  6. [Epoch Sync] EpochSyncRequest should not be a routed message nearcore#12091
  7. [Epoch Sync] How to ensure block headers are available for transaction validity period check? nearcore#12142
  8. [Epoch Sync] Add config validation for various sync parameters nearcore#12227
  9. [Epoch Sync] Merkle tree size for a block mismatches with the block ordinal? nearcore#12256
  10. [Epoch Sync] Generate merkle proof for the first block of current epoch nearcore#12255
  11. [Epoch Sync] Missing epoch_sync_data_hash in final_block_header_in_next_epoch nearcore#12258
  12. [Epoch Sync] Verify 2 last block headers in prev epoch nearcore#12259
  13. [Epoch Sync] Do not trigger epoch sync unless head is at genesis nearcore#12309
  14. [Epoch Sync] Consider adding versioning to the epoch sync proof for future upgrade-ability. nearcore#12310
  15. [Epoch Sync] Test bootstrapping a mainnet node against another epoch-synced mainnet node nearcore#12311
  16. [Epoch Sync] Add tests for epoch sync proof verification nearcore#12260
  17. [Epoch Sync] Enable epoch sync config by default nearcore#12347

Nice-to-have tasks:

Tasks
Beta Give feedback
  1. [EpochSync] Defend against malicious peers sending invalid epoch sync nearcore#11976
  2. [Epoch Sync] Provide external documentation for how to use Epoch Sync nearcore#12348
  3. [Epoch Sync] Come up with a story for preserving historical headers nearcore#12349
  4. [Epoch Sync] Does RPC node need historical headers? nearcore#12351
  5. Header sync incorrectly disregards peers at startup nearcore#11930
  6. [Epoch Sync] Error in gc: DB Not Found Error: BLOCK HEADER: 11111111111111111111111111111111 nearcore#11936
  7. [Epoch Sync] Add end to end test for epoch sync nearcore#12228
  8. [Epoch Sync] Update store validator to correctly handle nodes with missing data due to epoch sync nearcore#12229

Deprioritized (unimportant) tasks

Tasks
Beta Give feedback
  1. [Epoch Sync] EpochInfoAggregator treatment is awkward at the epoch sync boundary nearcore#11931
  2. [Epoch Sync] Minor: Header sync progress is misleading after epoch sync nearcore#11938
  3. [Epoch Sync] Ensure that syncing to block header < V3 is unsupported nearcore#12129
@robin-near robin-near self-assigned this Aug 13, 2024
This was referenced Aug 14, 2024
Merged
Closed
@robin-near robin-near changed the title [ProjectTracking]: Epoch Sync [ProjectTracking]: Epoch Sync V4 Aug 20, 2024
github-merge-queue bot pushed a commit to near/nearcore that referenced this issue Aug 22, 2024
@robin-near
[Epoch Sync] Initial implementation for Epoch Sync V4. (#11934)
3a9c72a 
Remaining issues:

near/near-one-project-tracking#73

I still need to go through this one more time, but reviews can be done
in the meantime.
@staffik staffik mentioned this issue Oct 18, 2024
Merged
github-merge-queue bot pushed a commit to near/nearcore that referenced this issue Oct 18, 2024
@staffik
[Epoch Sync] EpochSyncProof validation (#12020)
2c40e53 
Issue: #11932

Implements EpochSyncProof validation, according to:
- [design
doc](https://docs.google.com/document/d/14Itc9Hs7ewTRmcGANid9UCaYcJzaLzzM7FsvYYIFKDY/edit#heading=h.p2oiicluwd6o)
-
[`epoch_sync.rs`](https://github.com/near/nearcore/blob/d6d6e4782f7d57a7b7acc2894e1668355012ff74/core/primitives/src/epoch_sync.rs)

Summary:
- Move block merkle proof retrieval logic from `Chain` to a separate
trait (`MerkleProofAccess`). Make `Store` implements this trait so that
we can `get_block_proof` while serving `EpochSyncRequest` from a
separate thread and having no access to `Chain`.
- Refactor `verify_approval` and `compute_bp_hash` so that we can use it
for proof validation.
- Derive `first_block_info_in_epoch` from block header.
- Implement the validation logic.
- Add 2 more fields to the proof
  - `EpochSyncProofPastEpochData::protocol_version`
- `merkle_proof_for_first_block` generated at height of final block of
current epoch.
- Apparently, we need to add even more to enable generating
`merkle_proof_for_first_block` at the bootstrapped node. That will be
done as a follow-up issue:
#12255.
- Create a bunch of follow-up issues (added to the [tracking
issue](near/near-one-project-tracking#73)):
  - #12255
  - #12256
  - #12258
  - #12259
  - #12260
@staffik staffik mentioned this issue Nov 5, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@robin-near robin-near

Labels
project-tracking
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@robin-near