Backport changes from polakdot-sdk #2920
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Part of paritytech/polkadot-sdk#226 Related paritytech/polkadot-sdk#1833 - Deprecate `CurrencyAdapter` and introduce `FungibleAdapter` - Deprecate `ToStakingPot` and replace usage with `ResolveTo` - Required creating a new `StakingPotAccountId` struct that implements `TypedGet` for the staking pot account ID - Update parachain common utils `DealWithFees`, `ToAuthor` and `AssetsToBlockAuthor` implementations to use `fungible` - Update runtime XCM Weight Traders to use `ResolveTo` instead of `ToStakingPot` - Update runtime Transaction Payment pallets to use `FungibleAdapter` instead of `CurrencyAdapter` - [x] Blocked by paritytech/polkadot-sdk#1296, needs the `Unbalanced::decrease_balance` fix (cherry picked from commit bda4e75ac49786a7246531cf729b25c208cd38e6)
- What does this PR do?
1. Upgrades `trie-db`'s version to the latest release. This release
includes, among others, an implementation of `DoubleEndedIterator` for
the `TrieDB` struct, allowing to iterate both backwards and forwards
within the leaves of a trie.
2. Upgrades `trie-bench` to `0.39.0` for compatibility.
3. Upgrades `criterion` to `0.5.1` for compatibility.
- Why are these changes needed?
Besides keeping up with the upgrade of `trie-db`, this specifically adds
the functionality of iterating back on the leafs of a trie, with
`sp-trie`. In a project we're currently working on, this comes very
handy to verify a Merkle proof that is the response to a challenge. The
challenge is a random hash that (most likely) will not be an existing
leaf in the trie. So the challenged user, has to provide a Merkle proof
of the previous and next existing leafs in the trie, that surround the
random challenged hash.
Without having DoubleEnded iterators, we're forced to iterate until we
find the first existing leaf, like so:
```rust
// ************* VERIFIER (RUNTIME) *************
// Verify proof. This generates a partial trie based on the proof and
// checks that the root hash matches the `expected_root`.
let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap();
let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build();
// Print all leaf node keys and values.
println!("\nPrinting leaf nodes of partial tree...");
for key in trie.key_iter().unwrap() {
if key.is_ok() {
println!("Leaf node key: {:?}", key.clone().unwrap());
let val = trie.get(&key.unwrap());
if val.is_ok() {
println!("Leaf node value: {:?}", val.unwrap());
} else {
println!("Leaf node value: None");
}
}
}
println!("RECONSTRUCTED TRIE {:#?}", trie);
// Create an iterator over the leaf nodes.
let mut iter = trie.iter().unwrap();
// First element with a value should be the previous existing leaf to the challenged hash.
let mut prev_key = None;
for element in &mut iter {
if element.is_ok() {
let (key, _) = element.unwrap();
prev_key = Some(key);
break;
}
}
assert!(prev_key.is_some());
// Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly.
assert!(prev_key.unwrap() <= challenge_hash.to_vec());
// The next element should exist (meaning there is no other existing leaf between the
// previous and next leaf) and it should be greater than the challenged hash.
let next_key = iter.next().unwrap().unwrap().0;
assert!(next_key >= challenge_hash.to_vec());
```
With DoubleEnded iterators, we can avoid that, like this:
```rust
// ************* VERIFIER (RUNTIME) *************
// Verify proof. This generates a partial trie based on the proof and
// checks that the root hash matches the `expected_root`.
let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap();
let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build();
// Print all leaf node keys and values.
println!("\nPrinting leaf nodes of partial tree...");
for key in trie.key_iter().unwrap() {
if key.is_ok() {
println!("Leaf node key: {:?}", key.clone().unwrap());
let val = trie.get(&key.unwrap());
if val.is_ok() {
println!("Leaf node value: {:?}", val.unwrap());
} else {
println!("Leaf node value: None");
}
}
}
// println!("RECONSTRUCTED TRIE {:#?}", trie);
println!("\nChallenged key: {:?}", challenge_hash);
// Create an iterator over the leaf nodes.
let mut double_ended_iter = trie.into_double_ended_iter().unwrap();
// First element with a value should be the previous existing leaf to the challenged hash.
double_ended_iter.seek(&challenge_hash.to_vec()).unwrap();
let next_key = double_ended_iter.next_back().unwrap().unwrap().0;
let prev_key = double_ended_iter.next_back().unwrap().unwrap().0;
// Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly.
println!("Prev key: {:?}", prev_key);
assert!(prev_key <= challenge_hash.to_vec());
println!("Next key: {:?}", next_key);
assert!(next_key >= challenge_hash.to_vec());
```
- How were these changes implemented and what do they affect?
All that is needed for this functionality to be exposed is changing the
version number of `trie-db` in all the `Cargo.toml`s applicable, and
re-exporting some additional structs from `trie-db` in `sp-trie`.
---------
Co-authored-by: Bastian Köcher <[email protected]>
(cherry picked from commit 4e73c0fcd37e4e8c14aeb83b5c9e680981e16079)
bkontur
approved these changes
Apr 9, 2024
acatangiu
approved these changes
Apr 9, 2024
serban300
added a commit
that referenced
this pull request
Apr 9, 2024
* Migrate fee payment from `Currency` to `fungible` (#2292) Part of paritytech/polkadot-sdk#226 Related paritytech/polkadot-sdk#1833 - Deprecate `CurrencyAdapter` and introduce `FungibleAdapter` - Deprecate `ToStakingPot` and replace usage with `ResolveTo` - Required creating a new `StakingPotAccountId` struct that implements `TypedGet` for the staking pot account ID - Update parachain common utils `DealWithFees`, `ToAuthor` and `AssetsToBlockAuthor` implementations to use `fungible` - Update runtime XCM Weight Traders to use `ResolveTo` instead of `ToStakingPot` - Update runtime Transaction Payment pallets to use `FungibleAdapter` instead of `CurrencyAdapter` - [x] Blocked by paritytech/polkadot-sdk#1296, needs the `Unbalanced::decrease_balance` fix (cherry picked from commit bda4e75ac49786a7246531cf729b25c208cd38e6) * Upgrade `trie-db` from `0.28.0` to `0.29.0` (#3982) - What does this PR do? 1. Upgrades `trie-db`'s version to the latest release. This release includes, among others, an implementation of `DoubleEndedIterator` for the `TrieDB` struct, allowing to iterate both backwards and forwards within the leaves of a trie. 2. Upgrades `trie-bench` to `0.39.0` for compatibility. 3. Upgrades `criterion` to `0.5.1` for compatibility. - Why are these changes needed? Besides keeping up with the upgrade of `trie-db`, this specifically adds the functionality of iterating back on the leafs of a trie, with `sp-trie`. In a project we're currently working on, this comes very handy to verify a Merkle proof that is the response to a challenge. The challenge is a random hash that (most likely) will not be an existing leaf in the trie. So the challenged user, has to provide a Merkle proof of the previous and next existing leafs in the trie, that surround the random challenged hash. Without having DoubleEnded iterators, we're forced to iterate until we find the first existing leaf, like so: ```rust // ************* VERIFIER (RUNTIME) ************* // Verify proof. This generates a partial trie based on the proof and // checks that the root hash matches the `expected_root`. let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap(); let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build(); // Print all leaf node keys and values. println!("\nPrinting leaf nodes of partial tree..."); for key in trie.key_iter().unwrap() { if key.is_ok() { println!("Leaf node key: {:?}", key.clone().unwrap()); let val = trie.get(&key.unwrap()); if val.is_ok() { println!("Leaf node value: {:?}", val.unwrap()); } else { println!("Leaf node value: None"); } } } println!("RECONSTRUCTED TRIE {:#?}", trie); // Create an iterator over the leaf nodes. let mut iter = trie.iter().unwrap(); // First element with a value should be the previous existing leaf to the challenged hash. let mut prev_key = None; for element in &mut iter { if element.is_ok() { let (key, _) = element.unwrap(); prev_key = Some(key); break; } } assert!(prev_key.is_some()); // Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly. assert!(prev_key.unwrap() <= challenge_hash.to_vec()); // The next element should exist (meaning there is no other existing leaf between the // previous and next leaf) and it should be greater than the challenged hash. let next_key = iter.next().unwrap().unwrap().0; assert!(next_key >= challenge_hash.to_vec()); ``` With DoubleEnded iterators, we can avoid that, like this: ```rust // ************* VERIFIER (RUNTIME) ************* // Verify proof. This generates a partial trie based on the proof and // checks that the root hash matches the `expected_root`. let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap(); let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build(); // Print all leaf node keys and values. println!("\nPrinting leaf nodes of partial tree..."); for key in trie.key_iter().unwrap() { if key.is_ok() { println!("Leaf node key: {:?}", key.clone().unwrap()); let val = trie.get(&key.unwrap()); if val.is_ok() { println!("Leaf node value: {:?}", val.unwrap()); } else { println!("Leaf node value: None"); } } } // println!("RECONSTRUCTED TRIE {:#?}", trie); println!("\nChallenged key: {:?}", challenge_hash); // Create an iterator over the leaf nodes. let mut double_ended_iter = trie.into_double_ended_iter().unwrap(); // First element with a value should be the previous existing leaf to the challenged hash. double_ended_iter.seek(&challenge_hash.to_vec()).unwrap(); let next_key = double_ended_iter.next_back().unwrap().unwrap().0; let prev_key = double_ended_iter.next_back().unwrap().unwrap().0; // Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly. println!("Prev key: {:?}", prev_key); assert!(prev_key <= challenge_hash.to_vec()); println!("Next key: {:?}", next_key); assert!(next_key >= challenge_hash.to_vec()); ``` - How were these changes implemented and what do they affect? All that is needed for this functionality to be exposed is changing the version number of `trie-db` in all the `Cargo.toml`s applicable, and re-exporting some additional structs from `trie-db` in `sp-trie`. --------- Co-authored-by: Bastian Köcher <[email protected]> (cherry picked from commit 4e73c0fcd37e4e8c14aeb83b5c9e680981e16079) * Update polkadot-sdk refs * Fix Cargo.lock --------- Co-authored-by: Liam Aharon <[email protected]> Co-authored-by: Facundo Farall <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Keeping the repo up to date with polkadot-sdk until the migration PR is merged