feat(accountsdb,gossip): Initial snapshot propagation work #333

1. Guard the full and incremental snapshot info under the same rwmux, both to deduplicate the full slot value, and to synchronize the existence of incremental snapshots relative to full snapshots. 2. Also do the same with the first snapshot info.

* Grouped fields into a few categories. * Change init to take only an `InitParams` struct. * Lowercase `AllocatorConfig` fields. * Share the `AllocatorConfig` enum tag across a few different places. * Inline `InitConfig`'s fields and remove it. * Fix code that potentially leaked a disk allocator pointer.

It was incorrectly reporting `file_info.length` as the file size, when it should have been using `account_file.memory.len`. This also adds a couple more runtime checks to assert the total written bytes are aligned to full 512 byte blocks.

It's currently disabled because it overwrites the snapshot archives in the test data dir; can comment the skip statement in order to test it locally for now. As the TODO says, we need a mechanism for loading from a snapshot which isn't in the accountsdb's snapshot working directory. This also adds a simple way to test the equality of the `SnapshotHashes` structure in tests.

The potential memory savings were minimal at best, and came at a significant cost in complexity - a simple tagged union is sufficient to avoid the overhead of an allocation, whilst retaining simplicity across the rest of the code (especially wrt generic comparisons).

Sequence it before deinit

In the future there should be a better API for loading from a snapshot that's outside the accountsdb snapshot working directory.

* Refactor `initSigned`, and related init methods, and call sites. * Move methods into more scoped types. * Flatten the `Gossip` struct into `AccountsDB`, and only store `my_pubkey`, instead of the whole key pair. * Change `push_msg_queue` to queue unsigned gossip data, and then sign it in `drainPushQueueToGossipTable`. * Set the wallclock time in `drainPushQueueToGossipTable` as well.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(accountsdb,gossip): Initial snapshot propagation work #333

feat(accountsdb,gossip): Initial snapshot propagation work #333

Commits on Nov 6, 2024