Rewrite db & storage

[yaziciahmet]

Description

Partial rewrite of the storage system. Here is an overview of changes made:

• No more snapshots. Snapshots were used to handle forks, but we don't deal with that at all at the moment, and give ourselves a finalization margin. The only "snapshot"-like behavior we need is to be able to be query and replay on top of a previous l2 block. To do that, all we need is to use the right version when querying the underlying database.
• Current hierarchy of storage structures goes as following: WorkingSet -> ProverStorage -> StateDB + NativeDB -> DbTransaction -> DB. DbTransaction is simply a structure that stores local writes in its cache, and queries cache+db whenever get is called. When freezed, returns uncommitted changes to be committed to underlying database.
• StateDB does not have a version in its state anymore, and it completely leaves the version to use to caller. It has an helper method to return the next expected version.
• ProverStorage is the main structure that maintains a version. For every query to database ProverStorage does, it uses its own internal version, and its api no longer accepts version. This simplifies the version flow of storage. ProverStorage also has a flag called is_snapshot. If it is marked as snapshot, it can not be committed to underlying rocksdb, and finalization will panic (to enforce us devs to never do this and catch it on development)
• ProverStorage version indicates which version to query by. But commits will happen to version + 1. So it is like, storage is on version 2 gives the view of version 2 (including 2), and the next commit will be for version 3.
• ProverStorageManager's only job is to create requested ProverStorage on requested version. It has 3 methods to create storage: 1. create_latest_version_storage: creates a prover storage with the latest used version. used when nodes want to run or generate the next block. 2. create_latest_view_storage: creates a prover storage which always returns the latest version values without needing to update the version. this is achieved by specifying version as u64::MAX. this creates a snapshot storage, hence, can not be finalized. this is used by rpc and evm db provider. 3. create_storage_snapshot_on_version: creates a prover storage on the given version. it creates the storage as snapshot, and is useful for querying or replaying blocks on top of older blocks.
• Rpc context has a latest view storage, so it always gets the latest version. When rpcs want to replay transactions, for getting traces, there is a helper method called clone_with_version on ProverStorage, which returns a snapshot storage with the provided version. So any replay can be done without affecting the original ProverStorage.

One side-note that is rather important is that, before, we were committing state db versions as block + 1. so the block 1 would be version 2 in the db. But weirdly, native db was such that block 1 is committed as version 1. This pr changes that and makes native db block 1 to be version 2 (+1). IT IS BREAKING CURRENT NODES. But it is stupid to have such difference, and I think it is something worth to resync native db.

Linked Issues

• Fixes # (issue, if applicable)
• Related to # (issue)

Testing

Describe how these changes were tested. If you've added new features, have you added unit tests?

Docs

Describe where this code is documented. If it changes a documented interface, have the docs been updated?

[eyusufatik]

not really a detailed review just peeked over the changes

[codecov]

Codecov Report

Attention: Patch coverage is 96.52174% with 16 lines in your changes missing coverage. Please review.

Project coverage is 75.7%. Comparing base (f2bbd86) to head (12c2f28).
Report is 1 commits behind head on nightly.

Files with missing lines	Patch %	Lines
...-sdk/module-system/sov-state/src/prover_storage.rs	93.2%	4 Missing ⚠️
...eign-sdk/module-system/sov-state/src/zk_storage.rs	0.0%	4 Missing ⚠️
bin/cli/src/commands/prune.rs	0.0%	2 Missing ⚠️
crates/batch-prover/src/runner.rs	83.3%	1 Missing ⚠️
crates/fullnode/src/runner.rs	83.3%	1 Missing ⚠️
crates/sequencer/src/runner.rs	87.5%	1 Missing ⚠️
...-sdk/full-node/db/sov-schema-db/src/transaction.rs	99.4%	1 Missing ⚠️
...dk/full-node/sov-prover-storage-manager/src/lib.rs	97.9%	1 Missing ⚠️
.../module-system/sov-modules-core/src/storage/mod.rs	0.0%	1 Missing ⚠️

Additional details and impacted files

Files with missing lines	Coverage Δ
bin/citrea/src/eth.rs	100.0% <100.0%> (ø)
bin/citrea/src/main.rs	0.0% <ø> (ø)
bin/citrea/src/rollup/bitcoin.rs	0.0% <ø> (ø)
bin/citrea/src/rollup/mock.rs	76.0% <100.0%> (ø)
bin/citrea/src/rollup/mod.rs	79.6% <100.0%> (-1.2%)	⬇️
crates/batch-prover/src/lib.rs	100.0% <100.0%> (ø)
crates/common/src/backup/manager.rs	20.6% <100.0%> (ø)
crates/fullnode/src/lib.rs	100.0% <100.0%> (ø)
crates/sequencer/src/lib.rs	100.0% <100.0%> (ø)
...sovereign-sdk/full-node/db/sov-db/src/native_db.rs	100.0% <100.0%> (ø)
... and 19 more

... and 7 files with indirect coverage changes

[eyusufatik]

small nits. well done.