Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: caching of recently used coins #1105

Merged
merged 55 commits into from
Nov 6, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
80bca9f
initial cache setup
MujkicA Aug 28, 2023
14b3bef
Delete cache.patch
MujkicA Aug 28, 2023
60f62ed
remove .into on chain id
MujkicA Aug 29, 2023
ca0834b
Merge branch 'feature/utxo_caching' of github.com:FuelLabs/fuels-rs i…
MujkicA Aug 29, 2023
65a5f9c
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 2, 2023
3f4e2b4
improve adjusting for fee
MujkicA Oct 3, 2023
860429d
adapt account trait
MujkicA Oct 3, 2023
68a2751
update tests
MujkicA Oct 3, 2023
0abc057
refactor
MujkicA Oct 17, 2023
f21fb25
add time based cache
MujkicA Oct 18, 2023
2e1912d
fix coin cache expiry
MujkicA Oct 18, 2023
f356e6a
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 18, 2023
5066419
fix build
MujkicA Oct 18, 2023
751e1d0
use set for used coins
MujkicA Oct 19, 2023
273215b
use new ttl cache impl
MujkicA Oct 19, 2023
2a06808
improve tests and add feature flag
MujkicA Oct 20, 2023
b3c0e11
cleanup code
MujkicA Oct 21, 2023
5ad84f8
more cleanup
MujkicA Oct 21, 2023
92adfa9
uncomment test
MujkicA Oct 21, 2023
b31e235
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 21, 2023
99d9452
correct import path
MujkicA Oct 21, 2023
67b22a6
correct import path again
MujkicA Oct 21, 2023
5274ce9
wasm import path
MujkicA Oct 21, 2023
3c79570
fix no std on coin type id
MujkicA Oct 23, 2023
59cbdd4
add coin id file
MujkicA Oct 23, 2023
18cecd8
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 23, 2023
7243eba
fix imports
MujkicA Oct 23, 2023
5f5e83c
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 24, 2023
71dca1d
fix after merge
MujkicA Oct 24, 2023
6e4b67d
Update packages/fuels-programs/src/contract.rs
MujkicA Oct 30, 2023
c359a8d
Update packages/fuels-accounts/src/provider.rs
MujkicA Oct 30, 2023
fc518b1
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 30, 2023
7634e80
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 30, 2023
b82e4db
Update packages/fuels-core/src/types/wrappers/transaction.rs
MujkicA Oct 30, 2023
0895e57
Update packages/fuels-accounts/src/account.rs
MujkicA Oct 30, 2023
aa62fa3
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 30, 2023
9c9f20b
Update packages/fuels-accounts/src/account.rs
MujkicA Oct 30, 2023
ae8ecbc
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 30, 2023
47664a8
Update packages/fuels-core/src/types/transaction_builders.rs
MujkicA Oct 30, 2023
b65c591
Update packages/fuels-accounts/src/accounts_utils.rs
MujkicA Oct 30, 2023
bb71270
Merge branch 'master' into feature/utxo_caching
hal3e Oct 30, 2023
5a40bd3
add review suggestions
MujkicA Oct 30, 2023
b71d6ac
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 30, 2023
90e39ec
Merge branch 'master' into feature/utxo_caching
MujkicA Oct 30, 2023
7d32519
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 31, 2023
261ec1a
Update packages/fuels-accounts/src/coin_cache.rs
MujkicA Oct 31, 2023
9ce1823
Merge branch 'master' into feature/utxo_caching
iqdecay Nov 1, 2023
ba43b00
add rev comments
MujkicA Nov 2, 2023
7f0cb56
invalidate on failed tx
MujkicA Nov 2, 2023
dc94a77
Merge branch 'master' into feature/utxo_caching
MujkicA Nov 2, 2023
51f7fcc
fix cache test
MujkicA Nov 3, 2023
cad8313
make cache feature default
MujkicA Nov 3, 2023
c28aa96
use coinCacheKey
MujkicA Nov 6, 2023
ad07ddd
use loop instead of for each
MujkicA Nov 6, 2023
00c994b
Merge branch 'master' into feature/utxo_caching
digorithm Nov 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,7 @@ jobs:
args: --all-targets --features "default fuel-core-lib test-type-paths"
download_sway_artifacts: sway-examples-w-type-paths
- cargo_command: nextest
args: run --all-targets --features "default fuel-core-lib test-type-paths" --workspace
args: run --all-targets --features "default fuel-core-lib test-type-paths coin-cache" --workspace
download_sway_artifacts: sway-examples-w-type-paths
install_fuel_core: true
- cargo_command: nextest
Expand Down
2 changes: 2 additions & 0 deletions packages/fuels-accounts/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@ zeroize = { workspace = true, features = ["derive"] }

[dev-dependencies]
tempfile = { workspace = true }
tokio = { workspace = true, features = ["test-util"]}

[features]
default = ["std"]
coin-cache = ["tokio?/time"]
std = ["fuels-core/std", "dep:tokio", "fuel-core-client/default", "dep:eth-keystore"]
103 changes: 56 additions & 47 deletions packages/fuels-accounts/src/account.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ use fuels_core::{
};

use crate::{
accounts_utils::extract_message_id,
accounts_utils::{adjust_inputs_outputs, calculate_missing_base_amount, extract_message_id},
provider::{Provider, ResourceFilter},
};

Expand Down Expand Up @@ -117,26 +117,6 @@ pub trait ViewOnlyAccount: std::fmt::Debug + Send + Sync + Clone {
.await
.map_err(Into::into)
}

// /// Get some spendable resources (coins and messages) of asset `asset_id` owned by the account
// /// that add up at least to amount `amount`. The returned coins (UTXOs) are actual coins that
// /// can be spent. The number of UXTOs is optimized to prevent dust accumulation.
async fn get_spendable_resources(
&self,
asset_id: AssetId,
amount: u64,
) -> Result<Vec<CoinType>> {
let filter = ResourceFilter {
from: self.address().clone(),
asset_id,
amount,
..Default::default()
};
self.try_provider()?
.get_spendable_resources(filter)
.await
.map_err(Into::into)
}
}

#[cfg_attr(not(target_arch = "wasm32"), async_trait)]
Expand Down Expand Up @@ -166,11 +146,50 @@ pub trait Account: ViewOnlyAccount {
]
}

async fn add_fee_resources<Tb: TransactionBuilder>(
/// Get some spendable resources (coins and messages) of asset `asset_id` owned by the account
/// that add up at least to amount `amount`. The returned coins (UTXOs) are actual coins that
/// can be spent. The number of UXTOs is optimized to prevent dust accumulation.
MujkicA marked this conversation as resolved.
Show resolved Hide resolved
async fn get_spendable_resources(
&self,
tb: Tb,
previous_base_amount: u64,
) -> Result<Tb::TxType>;
asset_id: AssetId,
amount: u64,
) -> Result<Vec<CoinType>> {
let filter = ResourceFilter {
from: self.address().clone(),
asset_id,
amount,
..Default::default()
};

self.try_provider()?
.get_spendable_resources(filter)
.await
.map_err(Into::into)
}

/// Add base asset inputs to the transaction to cover the estimated fee.
/// Requires contract inputs to be at the start of the transactions inputs vec
/// so that their indexes are retained
async fn adjust_for_fee<Tb: TransactionBuilder>(
&self,
tb: &mut Tb,
used_base_amount: u64,
) -> Result<()> {
let missing_base_amount = calculate_missing_base_amount(tb, used_base_amount)?;

if missing_base_amount > 0 {
let new_base_inputs = self
.get_asset_inputs_for_amount(BASE_ASSET_ID, missing_base_amount)
.await?;

adjust_inputs_outputs(tb, new_base_inputs, self.address());
};

Ok(())
}

// Add signatures to the builder if the underlying account is a wallet
fn add_witnessses<Tb: TransactionBuilder>(&self, _tb: &mut Tb) {}

/// Transfer funds from this account to another `Address`.
/// Fails if amount for asset ID is larger than address's spendable coins.
Expand All @@ -186,26 +205,19 @@ pub trait Account: ViewOnlyAccount {
let network_info = provider.network_info().await?;

let inputs = self.get_asset_inputs_for_amount(asset_id, amount).await?;

let outputs = self.get_asset_outputs_for_amount(to, asset_id, amount);

let tx_builder = ScriptTransactionBuilder::prepare_transfer(
let mut tx_builder = ScriptTransactionBuilder::prepare_transfer(
inputs,
outputs,
tx_parameters,
network_info,
);

// if we are not transferring the base asset, previous base amount is 0
let previous_base_amount = if asset_id == AssetId::default() {
amount
} else {
0
};
self.add_witnessses(&mut tx_builder);
self.adjust_for_fee(&mut tx_builder, amount).await?;

let tx = self
.add_fee_resources(tx_builder, previous_base_amount)
.await?;
let tx = tx_builder.build()?;
let tx_id = provider.send_transaction_and_await_commit(tx).await?;

let receipts = provider
Expand Down Expand Up @@ -254,7 +266,7 @@ pub trait Account: ViewOnlyAccount {
];

// Build transaction and sign it
let tb = ScriptTransactionBuilder::prepare_contract_transfer(
let mut tb = ScriptTransactionBuilder::prepare_contract_transfer(
plain_contract_id,
balance,
asset_id,
Expand All @@ -264,14 +276,9 @@ pub trait Account: ViewOnlyAccount {
network_info,
);

// if we are not transferring the base asset, previous base amount is 0
let base_amount = if asset_id == AssetId::default() {
balance
} else {
0
};

let tx = self.add_fee_resources(tb, base_amount).await?;
self.add_witnessses(&mut tb);
self.adjust_for_fee(&mut tb, balance).await?;
let tx = tb.build()?;

let tx_id = provider.send_transaction_and_await_commit(tx).await?;

Expand Down Expand Up @@ -299,15 +306,17 @@ pub trait Account: ViewOnlyAccount {
.get_asset_inputs_for_amount(BASE_ASSET_ID, amount)
.await?;

let tb = ScriptTransactionBuilder::prepare_message_to_output(
let mut tb = ScriptTransactionBuilder::prepare_message_to_output(
to.into(),
amount,
inputs,
tx_parameters,
network_info,
);

let tx = self.add_fee_resources(tb, amount).await?;
self.add_witnessses(&mut tb);
self.adjust_for_fee(&mut tb, amount).await?;
let tx = tb.build()?;
let tx_id = provider.send_transaction_and_await_commit(tx).await?;

let receipts = provider
Expand Down
70 changes: 32 additions & 38 deletions packages/fuels-accounts/src/accounts_utils.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,73 +6,67 @@ use fuels_core::{
bech32::Bech32Address,
errors::{error, Error, Result},
input::Input,
transaction_builders::{NetworkInfo, TransactionBuilder},
transaction_builders::TransactionBuilder,
},
};

pub fn extract_message_id(receipts: &[Receipt]) -> Option<MessageId> {
receipts.iter().find_map(|m| m.message_id())
}

pub fn calculate_base_amount_with_fee(
pub fn calculate_missing_base_amount(
tb: &impl TransactionBuilder,
network_info: &NetworkInfo,
previous_base_amount: u64,
used_base_amount: u64,
) -> Result<u64> {
let transaction_fee = tb
.fee_checked_from_tx(network_info)?
.fee_checked_from_tx()?
.ok_or(error!(InvalidData, "Error calculating TransactionFee"))?;

let mut new_base_amount = transaction_fee.max_fee() + previous_base_amount;
let available_amount = available_base_amount(tb);

// If the tx doesn't consume any UTXOs, attempting to repeat it will lead to an
// error due to non unique tx ids (e.g. repeated contract call with configured gas cost of 0).
// Here we enforce a minimum amount on the base asset to avoid this
let is_consuming_utxos = tb
.inputs()
.iter()
.any(|input| !matches!(input, Input::Contract { .. }));
const MIN_AMOUNT: u64 = 1;
if !is_consuming_utxos && new_base_amount == 0 {
new_base_amount = MIN_AMOUNT;
}
let total_used = transaction_fee.max_fee() + used_base_amount;
let missing_amount = if total_used > available_amount {
total_used - available_amount
} else if !is_consuming_utxos(tb) {
// A tx needs to have at least 1 spendable input
// Enforce a minimum required amount on the base asset if no other inputs are present
1
} else {
0
};

Ok(new_base_amount)
Ok(missing_amount)
}

// Replace the current base asset inputs of a tx builder with the provided ones.
// Only signed resources and coin predicates are replaced, the remaining inputs are kept.
// Messages that contain data are also kept since we don't know who will consume the data.
pub fn adjust_inputs(
tb: &mut impl TransactionBuilder,
new_base_inputs: impl IntoIterator<Item = Input>,
) {
let adjusted_inputs = tb
.inputs()
fn available_base_amount(tb: &impl TransactionBuilder) -> u64 {
tb.inputs()
.iter()
.filter(|input| {
input.contains_data()
|| !matches!(input , Input::ResourceSigned { resource , .. }
| Input::ResourcePredicate { resource, .. } if resource.asset_id() == BASE_ASSET_ID)
.filter_map(|input| match (input.amount(), input.asset_id()) {
(Some(amount), Some(asset_id)) if asset_id == BASE_ASSET_ID => Some(amount),
_ => None,
})
.cloned()
.chain(new_base_inputs)
.collect();
.sum()
}

*tb.inputs_mut() = adjusted_inputs
fn is_consuming_utxos(tb: &impl TransactionBuilder) -> bool {
tb.inputs()
.iter()
.any(|input| !matches!(input, Input::Contract { .. }))
}

pub fn adjust_outputs(
pub fn adjust_inputs_outputs(
tb: &mut impl TransactionBuilder,
new_base_inputs: impl IntoIterator<Item = Input>,
address: &Bech32Address,
new_base_amount: u64,
) {
tb.inputs_mut().extend(new_base_inputs);

let is_base_change_present = tb.outputs().iter().any(|output| {
matches!(output , Output::Change { asset_id , .. }
if asset_id == & BASE_ASSET_ID)
});

if !is_base_change_present && new_base_amount != 0 {
if !is_base_change_present {
tb.outputs_mut()
.push(Output::change(address.into(), 0, BASE_ASSET_ID));
}
Expand Down
Loading
Loading