port(storage): use the cached block height from the database at the t…

…ime of creation of the view (#2646) ## Linked Issues/PRs  - same as #2639, but for master ## Description  The most important changes include adding metadata support to the `GenericDatabase`, modifying type definitions to include height metadata, and updating various methods to use the new database structure. ### Database Structure Improvements: * [`crates/fuel-core/src/database.rs`](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898R28): Added metadata support to the `GenericDatabase` and updated type definitions to include height metadata. This includes changes to methods such as `new`, `in_memory`, and `latest_view`. [[1]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898R28) [[2]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898R64-L73) [[3]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898R83-R89) [[4]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L129-R138) [[5]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L144-R152) [[6]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L179-R190) [[7]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L190-R209) [[8]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L238-R259) [[9]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L257-R274) [[10]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L270-R290) [[11]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898R346-R369) [[12]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L344-R378) [[13]](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L354-R396) * [`crates/fuel-core/src/state.rs`](diffhunk://#diff-386ccf4eb5acb74a927c8c9849dfcc59aa78bcf800d9fd89f4e1a6e30f7199f0R33-R51): Updated type definitions for `IterableKeyValueView` and `KeyValueView` to include height metadata. Modified the `TransactableStorage` trait to use the new types. [[1]](diffhunk://#diff-386ccf4eb5acb74a927c8c9849dfcc59aa78bcf800d9fd89f4e1a6e30f7199f0R33-R51) [[2]](diffhunk://#diff-386ccf4eb5acb74a927c8c9849dfcc59aa78bcf800d9fd89f4e1a6e30f7199f0L64-R68) [[3]](diffhunk://#diff-386ccf4eb5acb74a927c8c9849dfcc59aa78bcf800d9fd89f4e1a6e30f7199f0L82-L89) ### Code Refactoring: * [`crates/fuel-core/src/state/generic_database.rs`](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L31-R60): Refactored the `GenericDatabase` struct to include metadata and updated various trait implementations to support the new structure. [[1]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L31-R60) [[2]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L67-R76) [[3]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L77-R86) [[4]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L96-R106) [[5]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L106-R116) [[6]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L139-R149) [[7]](diffhunk://#diff-304688e42224bb543ff683ccd9bc78e6432eea2e2868da28fc91ca441e38ea17L165-R202) ### Performance Improvements: * [`crates/fuel-core/src/database.rs`](diffhunk://#diff-c95a3d57a39feac7c8c2f3b193a24eec39e794413adc741df36450f9a4539898L144-R152): Improved the `latest_height` method to use metadata for fetching the latest height, enhancing performance. ### Test Updates: * [`crates/fuel-core/src/service/genesis/importer/import_task.rs`](diffhunk://#diff-954b9280f787dd90eda70a58c77f53b3543e8286462d498dcda45536ca7a175fL581-R587): Updated tests to use the new `KeyValueView` and `IterableKeyValueView` types with height metadata. ## Checklist - [ ] Breaking changes are clearly marked as such in the PR description and changelog - [ ] New behavior is reflected in tests - [ ] [The specification](https://github.com/FuelLabs/fuel-specs/) matches the implemented behavior (link update PR if changes are needed) ### Before requesting review - [ ] I have reviewed the code myself - [ ] I have created follow-up issues caused by this PR and linked them here ### After merging, notify other teams [Add or remove entries as needed] - [ ] [Rust SDK](https://github.com/FuelLabs/fuels-rs/) - [ ] [Sway compiler](https://github.com/FuelLabs/sway/) - [ ] [Platform documentation](https://github.com/FuelLabs/devrel-requests/issues/new?assignees=&labels=new+request&projects=&template=NEW-REQUEST.yml&title=%5BRequest%5D%3A+) (for out-of-organization contributors, the person merging the PR will do this) - [ ] Someone else? --------- Co-authored-by: Green Baneling <[email protected]> Co-authored-by: Andrea Cerone <[email protected]>
FuelLabs · Feb 11, 2025 · 8f0951e · 8f0951e
1 parent 44bd923
commit 8f0951e
Show file tree

Hide file tree

Showing 8 changed files with 139 additions and 67 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -13,6 +13,9 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 ### Added
 - [2150](https://github.com/FuelLabs/fuel-core/pull/2150): Upgraded `libp2p` to `0.54.1` and introduced `ConnectionLimiter` to limit pending incoming/outgoing connections.
 
+### Fixed
+- [2646](https://github.com/FuelLabs/fuel-core/pull/2646): Improved performance of fetching block height by caching it when the view is created.
+
 ### Changed
 
 - [2653](https://github.com/FuelLabs/fuel-core/pull/2653): Added cleaner error for wasm-executor upon failed deserialization.

diff --git a/crates/fuel-core/src/database.rs b/crates/fuel-core/src/database.rs
@@ -25,6 +25,7 @@ use crate::{
     },
 };
 use fuel_core_chain_config::TableEntry;
+pub use fuel_core_database::Error;
 use fuel_core_gas_price_service::common::fuel_core_storage_adapter::storage::GasPriceMetadata;
 use fuel_core_services::SharedMutex;
 use fuel_core_storage::{
@@ -60,17 +61,12 @@ use itertools::Itertools;
 use std::{
     borrow::Cow,
     fmt::Debug,
+    io::Empty,
     sync::Arc,
 };
-
-pub use fuel_core_database::Error;
 pub type Result<T> = core::result::Result<T, Error>;
 
 // TODO: Extract `Database` and all belongs into `fuel-core-database`.
-use crate::database::database_description::{
-    gas_price::GasPriceDatabase,
-    indexation_availability,
-};
 #[cfg(feature = "rocksdb")]
 use crate::state::{
     historical_rocksdb::{
@@ -84,6 +80,13 @@ use crate::state::{
         RocksDb,
     },
 };
+use crate::{
+    database::database_description::{
+        gas_price::GasPriceDatabase,
+        indexation_availability,
+    },
+    state::HeightType,
+};
 #[cfg(feature = "rocksdb")]
 use std::path::Path;
 
@@ -126,10 +129,13 @@ where
 }
 
 pub type Database<Description = OnChain, Stage = RegularStage<Description>> =
-    GenericDatabase<DataSource<Description, Stage>>;
-pub type OnChainIterableKeyValueView = IterableKeyValueView<ColumnType<OnChain>>;
-pub type OffChainIterableKeyValueView = IterableKeyValueView<ColumnType<OffChain>>;
-pub type RelayerIterableKeyValueView = IterableKeyValueView<ColumnType<Relayer>>;
+    GenericDatabase<DataSource<Description, Stage>, Empty>;
+pub type OnChainIterableKeyValueView =
+    IterableKeyValueView<ColumnType<OnChain>, HeightType<OnChain>>;
+pub type OffChainIterableKeyValueView =
+    IterableKeyValueView<ColumnType<OffChain>, HeightType<OffChain>>;
+pub type RelayerIterableKeyValueView =
+    IterableKeyValueView<ColumnType<Relayer>, HeightType<Relayer>>;
 
 pub type GenesisDatabase<Description = OnChain> = Database<Description, GenesisStage>;
 
@@ -141,7 +147,9 @@ impl OnChainIterableKeyValueView {
     }
 
     pub fn latest_height(&self) -> StorageResult<BlockHeight> {
-        self.maybe_latest_height()?.ok_or(not_found!("BlockHeight"))
+        self.metadata()
+            .cloned()
+            .ok_or_else(|| not_found!("Metadata"))
     }
 
     pub fn latest_block(&self) -> StorageResult<CompressedBlock> {
@@ -176,7 +184,10 @@ where
     Description: DatabaseDescription,
 {
     pub fn new(data_source: DataSourceType<Description>) -> Self {
-        GenesisDatabase::from_storage(DataSource::new(data_source, GenesisStage))
+        GenesisDatabase::from_storage_and_metadata(
+            DataSource::new(data_source, GenesisStage),
+            None,
+        )
     }
 }
 
@@ -187,12 +198,15 @@ where
         StorageInspect<MetadataTable<Description>, Error = StorageError>,
 {
     pub fn new(data_source: DataSourceType<Description>) -> Self {
-        let mut database = Self::from_storage(DataSource::new(
-            data_source,
-            RegularStage {
-                height: SharedMutex::new(None),
-            },
-        ));
+        let mut database = Self::from_storage_and_metadata(
+            DataSource::new(
+                data_source,
+                RegularStage {
+                    height: SharedMutex::new(None),
+                },
+            ),
+            Some(Empty::default()),
+        );
         let height = database
             .latest_height_from_metadata()
             .expect("Failed to get latest height during creation of the database");
@@ -235,14 +249,14 @@ where
     ) -> core::result::Result<GenesisDatabase<Description>, GenesisDatabase<Description>>
     {
         if !self.stage.height.lock().is_some() {
-            Ok(GenesisDatabase::new(self.into_inner().data))
+            Ok(GenesisDatabase::new(self.into_inner().0.data))
         } else {
             tracing::warn!(
                 "Converting regular database into genesis, \
                 while height is already set for `{}`",
                 Description::name()
             );
-            Err(GenesisDatabase::new(self.into_inner().data))
+            Err(GenesisDatabase::new(self.into_inner().0.data))
         }
     }
 }
@@ -254,7 +268,10 @@ where
 {
     pub fn in_memory() -> Self {
         let data = Arc::<MemoryStore<Description>>::new(MemoryStore::default());
-        Self::from_storage(DataSource::new(data, Stage::default()))
+        Self::from_storage_and_metadata(
+            DataSource::new(data, Stage::default()),
+            Some(Empty::default()),
+        )
     }
 
     #[cfg(feature = "rocksdb")]
@@ -267,7 +284,10 @@ where
         )?;
         let historical_db = HistoricalRocksDB::new(db, state_rewind_policy)?;
         let data = Arc::new(historical_db);
-        Ok(Self::from_storage(DataSource::new(data, Stage::default())))
+        Ok(Self::from_storage_and_metadata(
+            DataSource::new(data, Stage::default()),
+            None,
+        ))
     }
 }
 
@@ -323,16 +343,31 @@ where
 
         Ok(())
     }
+
+    fn latest_view_with_height(
+        &self,
+        height: Option<Description::Height>,
+    ) -> StorageResult<IterableKeyValueView<ColumnType<Description>, Description::Height>>
+    {
+        let view = self.inner_storage().data.latest_view()?;
+
+        let (view, _) = view.into_inner();
+        Ok(IterableKeyValueView::from_storage_and_metadata(
+            view, height,
+        ))
+    }
 }
 
 impl<Description> AtomicView for Database<Description>
 where
     Description: DatabaseDescription,
 {
-    type LatestView = IterableKeyValueView<ColumnType<Description>>;
+    type LatestView = IterableKeyValueView<ColumnType<Description>, Description::Height>;
 
     fn latest_view(&self) -> StorageResult<Self::LatestView> {
-        self.inner_storage().data.latest_view()
+        let lock = self.inner_storage().stage.height.lock();
+        let view = self.latest_view_with_height(*lock)?;
+        Ok(view)
     }
 }
 
@@ -341,7 +376,7 @@ where
     Description: DatabaseDescription,
 {
     type Height = Description::Height;
-    type ViewAtHeight = KeyValueView<ColumnType<Description>>;
+    type ViewAtHeight = KeyValueView<ColumnType<Description>, Description::Height>;
 
     fn latest_height(&self) -> Option<Self::Height> {
         *self.inner_storage().stage.height.lock()
@@ -351,9 +386,15 @@ where
         let lock = self.inner_storage().stage.height.lock();
 
         match *lock {
-            None => return self.latest_view().map(|view| view.into_key_value_view()),
+            None => {
+                return self
+                    .latest_view_with_height(None)
+                    .map(|view| view.into_key_value_view())
+            }
             Some(current_height) if &current_height == height => {
-                return self.latest_view().map(|view| view.into_key_value_view())
+                return self
+                    .latest_view_with_height(Some(current_height))
+                    .map(|view| view.into_key_value_view())
             }
             _ => {}
         };

diff --git a/crates/fuel-core/src/database/storage.rs b/crates/fuel-core/src/database/storage.rs
@@ -16,7 +16,7 @@ use fuel_core_storage::{
     StorageWrite,
 };
 
-impl<Storage, M> StorageMutate<M> for GenericDatabase<Storage>
+impl<Storage, M, Metadata> StorageMutate<M> for GenericDatabase<Storage, Metadata>
 where
     M: Mappable,
     Self: Modifiable,
@@ -50,7 +50,7 @@ where
     }
 }
 
-impl<Storage, M> StorageWrite<M> for GenericDatabase<Storage>
+impl<Storage, M, Metadata> StorageWrite<M> for GenericDatabase<Storage, Metadata>
 where
     M: Mappable,
     StructuredStorage<Storage>: StorageInspect<M, Error = StorageError>,
@@ -95,7 +95,7 @@ where
     }
 }
 
-impl<Storage, M> StorageBatchMutate<M> for GenericDatabase<Storage>
+impl<Storage, M, Metadata> StorageBatchMutate<M> for GenericDatabase<Storage, Metadata>
 where
     M: Mappable,
     StructuredStorage<Storage>: StorageInspect<M, Error = StorageError>,

diff --git a/crates/fuel-core/src/service/genesis/importer/import_task.rs b/crates/fuel-core/src/service/genesis/importer/import_task.rs
@@ -582,11 +582,13 @@ mod tests {
         fn view_at_height(
             &self,
             _: &BlockHeight,
-        ) -> StorageResult<KeyValueView<Self::Column>> {
+        ) -> StorageResult<KeyValueView<Self::Column, BlockHeight>> {
             Err(anyhow::anyhow!("I refuse to work!").into())
         }
 
-        fn latest_view(&self) -> StorageResult<IterableKeyValueView<Self::Column>> {
+        fn latest_view(
+            &self,
+        ) -> StorageResult<IterableKeyValueView<Self::Column, BlockHeight>> {
             Err(anyhow::anyhow!("I refuse to work!").into())
         }
 

diff --git a/crates/fuel-core/src/state.rs b/crates/fuel-core/src/state.rs
@@ -30,23 +30,25 @@ pub mod rocks_db;
 pub mod rocks_db_key_iterator;
 
 pub type ColumnType<Description> = <Description as DatabaseDescription>::Column;
+pub type HeightType<Description> = <Description as DatabaseDescription>::Height;
 
 /// A type extends the `KeyValueView`, allowing iteration over the storage.
-pub type IterableKeyValueView<Column> =
-    GenericDatabase<IterableKeyValueViewWrapper<Column>>;
+pub type IterableKeyValueView<Column, BlockHeight> =
+    GenericDatabase<IterableKeyValueViewWrapper<Column>, BlockHeight>;
 
 /// The basic view available for the key value storage.
-pub type KeyValueView<Column> = GenericDatabase<KeyValueViewWrapper<Column>>;
+pub type KeyValueView<Column, BlockHeight> =
+    GenericDatabase<KeyValueViewWrapper<Column>, BlockHeight>;
 
-impl<Column> IterableKeyValueView<Column>
+impl<Column, Height> IterableKeyValueView<Column, Height>
 where
     Column: StorageColumn + 'static,
 {
     /// Downgrades the `IterableKeyValueView` into the `KeyValueView`.
-    pub fn into_key_value_view(self) -> KeyValueView<Column> {
-        let iterable = self.into_inner();
+    pub fn into_key_value_view(self) -> KeyValueView<Column, Height> {
+        let (iterable, metadata) = self.into_inner();
         let storage = KeyValueViewWrapper::new(iterable);
-        KeyValueView::from_storage(storage)
+        KeyValueView::from_storage_and_metadata(storage, metadata)
     }
 }
 
@@ -61,9 +63,9 @@ pub trait TransactableStorage<Height>: IterableStore + Debug + Send + Sync {
     fn view_at_height(
         &self,
         height: &Height,
-    ) -> StorageResult<KeyValueView<Self::Column>>;
+    ) -> StorageResult<KeyValueView<Self::Column, Height>>;
 
-    fn latest_view(&self) -> StorageResult<IterableKeyValueView<Self::Column>>;
+    fn latest_view(&self) -> StorageResult<IterableKeyValueView<Self::Column, Height>>;
 
     fn rollback_block_to(&self, height: &Height) -> StorageResult<()>;
 }
@@ -79,14 +81,16 @@ where
         unimplemented!()
     }
 
-    fn view_at_height(&self, _: &Height) -> StorageResult<KeyValueView<Self::Column>> {
+    fn view_at_height(
+        &self,
+        _: &Height,
+    ) -> StorageResult<KeyValueView<Self::Column, Height>> {
         unimplemented!()
     }
 
-    fn latest_view(&self) -> StorageResult<IterableKeyValueView<Self::Column>> {
+    fn latest_view(&self) -> StorageResult<IterableKeyValueView<Self::Column, Height>> {
         unimplemented!()
     }
-
     fn rollback_block_to(&self, _: &Height) -> StorageResult<()> {
         unimplemented!()
     }