Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(tree): remove_blocks fixes, return hash and number in persistence task #10678

Merged
merged 1 commit into from
Sep 5, 2024

Conversation

Rjected
Copy link
Member

@Rjected Rjected commented Sep 3, 2024

This does many things:

  • Make remove_blocks_above and save_blocks output both a block number and hash.
  • Make remove_until take an upper bound block number, a last_persisted_block_num and hash, and an optional finalized_hash.
  • remove_until will now make sure that it only removes canonical in-memory blocks if they are in the same chain as the last persisted block. This was motivated by hive tests where the persistence task would run in between a reorg. In this case, we were removing canonical blocks that were not actually persisted yet.
  • remove_until is split into two methods, remove_canonical_until and prune_finalized_sidechains.
  • is_canonical is introduced as a helper for remove_until
  • remove_persisted_blocks is modified to only remove blocks if they are actually part of the persisted chain.
  • Blocks that have been reorged, but are not on-disk, are reinserted into the tree.
  • remove_blocks_above was incorrect, removing exclusively for some pieces of data, and inclusively for others. This is fixed.
  • There is now a VecDeque in case there are multiple reorgs in a row.

closes #10570

@Rjected Rjected force-pushed the dan/fix-hive-engine-api branch 6 times, most recently from 301fffe to ef55f7c Compare September 4, 2024 22:32
@Rjected Rjected force-pushed the dan/fix-hive-engine-api branch 11 times, most recently from 87051f8 to 9895b2d Compare September 4, 2024 23:13
@Rjected Rjected marked this pull request as ready for review September 4, 2024 23:14
@Rjected Rjected changed the title fix(tree): remove_blocks fixes, return number in persistence task fix(tree): remove_blocks fixes, return hash and number in persistence task Sep 4, 2024
Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only nits

I was able to follow and the vecque for queued disk reorgs makes sense

crates/engine/tree/src/persistence.rs Outdated Show resolved Hide resolved
@Rjected Rjected added this pull request to the merge queue Sep 5, 2024
Merged via the queue into main with commit 60bc403 Sep 5, 2024
34 checks passed
@Rjected Rjected deleted the dan/fix-hive-engine-api branch September 5, 2024 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Tracking: On-disk reorg handling
2 participants