Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [EXC-1768] Add system API to get costs of management canister calls. #3584

Open
wants to merge 84 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
9c07654
SystemAPI trait
michael-weigelt Jan 23, 2025
77c3096
SystemAPI trait changes
michael-weigelt Jan 28, 2025
14e8d66
implement key costs
michael-weigelt Jan 28, 2025
ba7f100
implement replication factor
michael-weigelt Jan 28, 2025
c18d911
implement cost_create_canister
michael-weigelt Jan 28, 2025
14e84a5
implement cost_call
michael-weigelt Jan 28, 2025
c4298fb
implement cost_http_request
michael-weigelt Jan 29, 2025
ed95b9a
fix ssss
michael-weigelt Jan 29, 2025
8613b76
clippy
michael-weigelt Jan 29, 2025
b50c819
clippy
michael-weigelt Jan 29, 2025
18a6aec
added signatures
michael-weigelt Jan 29, 2025
e087737
added replication factor linker
michael-weigelt Jan 29, 2025
f26f2bb
implement other linkers
michael-weigelt Jan 29, 2025
5e6c8b9
benchmarks
michael-weigelt Jan 29, 2025
552827c
enums
michael-weigelt Jan 29, 2025
bc3edbe
api availability
michael-weigelt Jan 29, 2025
63ac1cd
Merge remote-tracking branch 'origin' into mwe/costs
michael-weigelt Jan 29, 2025
a69255b
Update rs/interfaces/src/execution_environment.rs
michael-weigelt Feb 3, 2025
fcc7e41
Update rs/interfaces/src/execution_environment.rs
michael-weigelt Feb 3, 2025
1ff90d2
use full keyid for lookup
michael-weigelt Feb 4, 2025
a213b00
Merge remote-tracking branch 'origin' into mwe/costs
michael-weigelt Feb 4, 2025
f0f131f
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 4, 2025
b629e57
tryinto curves
michael-weigelt Feb 4, 2025
e3f6a0a
tryinto curves
michael-weigelt Feb 4, 2025
47e7ad5
conversion in system_api, not embedders
michael-weigelt Feb 4, 2025
936d0f0
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 4, 2025
81b8b70
Update rs/types/management_canister_types/src/lib.rs
michael-weigelt Feb 4, 2025
0b9a95e
Update rs/types/types/src/consensus/idkg/common.rs
michael-weigelt Feb 4, 2025
f9b5f5c
import
michael-weigelt Feb 4, 2025
8bb8a36
Merge branch 'mwe/costs', remote-tracking branch 'origin' into mwe/co…
michael-weigelt Feb 4, 2025
600d6ef
Merge remote-tracking branch 'origin' into mwe/costs
michael-weigelt Feb 4, 2025
b7825cd
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 4, 2025
1c7e712
validation
michael-weigelt Feb 4, 2025
e33abab
linkers
michael-weigelt Feb 4, 2025
035c113
benchmarks
michael-weigelt Feb 4, 2025
108dedc
naming
michael-weigelt Feb 4, 2025
f2c19f6
universal canister
michael-weigelt Feb 4, 2025
9ac4d8f
universal canister 2
michael-weigelt Feb 4, 2025
dbab533
copy paste error
michael-weigelt Feb 5, 2025
9bb4c65
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 5, 2025
588dc4b
fix tests
michael-weigelt Feb 5, 2025
c23811b
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 5, 2025
47df030
uni canister 3
michael-weigelt Feb 5, 2025
a649795
fix universal canister + test
michael-weigelt Feb 5, 2025
23d6bcb
Revert "implement replication factor"
michael-weigelt Feb 10, 2025
652b616
return types for cost api
michael-weigelt Feb 10, 2025
d44ca2d
Merge branch 'mwe/costs', remote-tracking branch 'origin' into mwe/co…
michael-weigelt Feb 10, 2025
f4a45d7
return values for sign cost endpoints
michael-weigelt Feb 10, 2025
419d290
Merge remote-tracking branch 'origin' into mwe/costs
michael-weigelt Feb 10, 2025
e633b83
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 10, 2025
3f20c23
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 10, 2025
12242a5
change sign signatures
michael-weigelt Feb 10, 2025
9a251e9
import fix
michael-weigelt Feb 10, 2025
9c31252
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 10, 2025
ee7d23d
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 10, 2025
ae184f7
cost_call test
michael-weigelt Feb 10, 2025
c243ebd
failing test
michael-weigelt Feb 10, 2025
13acb57
simplify test
michael-weigelt Feb 10, 2025
8791dec
zero cost config
michael-weigelt Feb 11, 2025
38bd751
fix implementation of cost call
michael-weigelt Feb 12, 2025
50781a5
fix tests
michael-weigelt Feb 12, 2025
3619b79
fix implementation of cost call
michael-weigelt Feb 12, 2025
61d0176
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 12, 2025
9ba9fea
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 12, 2025
85f6870
fix benchmarks
michael-weigelt Feb 12, 2025
913ec93
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 12, 2025
2346715
invoke calls tests
michael-weigelt Feb 12, 2025
6efe416
curve exhaustiveness tests
michael-weigelt Feb 12, 2025
796b911
improved description
michael-weigelt Feb 12, 2025
e63566b
doc
michael-weigelt Feb 12, 2025
ec7d9c6
curve exhaustiveness tests
michael-weigelt Feb 12, 2025
0d4d8d0
Merge branch 'mwe/costs' into mwe/costs2
michael-weigelt Feb 12, 2025
870848a
Merge branch 'mwe/costs2' into mwe/costs3
michael-weigelt Feb 12, 2025
f00810f
ecdsa tests
michael-weigelt Feb 12, 2025
95b25c9
schnorr tests
michael-weigelt Feb 12, 2025
598d83e
vetkd tests
michael-weigelt Feb 12, 2025
ec9d723
Merge branch 'master' into mwe/costs
michael-weigelt Feb 12, 2025
e41dd00
clippy
michael-weigelt Feb 12, 2025
6fe4aaa
Update rs/interfaces/src/execution_environment.rs
michael-weigelt Feb 14, 2025
07ca8fa
comments
michael-weigelt Feb 14, 2025
9c82316
comments
michael-weigelt Feb 14, 2025
539868c
Merge remote-tracking branch 'origin' into mwe/costs
michael-weigelt Feb 14, 2025
c91d02e
saturate
michael-weigelt Feb 17, 2025
1dd00ec
Merge branch 'master' into mwe/costs
michael-weigelt Feb 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion rs/canister_sandbox/src/sandbox_server.rs
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@ mod tests {
use ic_limits::SMALL_APP_SUBNET_MAX_SIZE;
use ic_logger::replica_logger::no_op_logger;
use ic_registry_subnet_type::SubnetType;
use ic_replicated_state::{Global, NumWasmPages, PageIndex, PageMap};
use ic_replicated_state::{Global, NetworkTopology, NumWasmPages, PageIndex, PageMap};
use ic_system_api::{
sandbox_safe_system_state::{CanisterStatusView, SandboxSafeSystemState},
ApiType, ExecutionParameters, InstructionLimits,
Expand Down Expand Up @@ -242,6 +242,7 @@ mod tests {
caller,
0,
IS_WASM64_EXECUTION,
NetworkTopology::default(),
)
}

Expand Down
101 changes: 100 additions & 1 deletion rs/interfaces/src/execution_environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ mod errors;
pub use errors::{CanisterBacktrace, CanisterOutOfCyclesError, HypervisorError, TrapCode};
use ic_base_types::NumBytes;
use ic_error_types::UserError;
use ic_management_canister_types::MasterPublicKeyId;
use ic_management_canister_types::{EcdsaCurve, MasterPublicKeyId, SchnorrAlgorithm, VetKdCurve};
use ic_registry_provisional_whitelist::ProvisionalWhitelist;
use ic_registry_subnet_type::SubnetType;
use ic_sys::{PageBytes, PageIndex};
Expand Down Expand Up @@ -1177,6 +1177,105 @@ pub trait SystemApi {
heap: &mut [u8],
) -> HypervisorResult<()>;

/// Returns the replication factor of the provided subnet.
///
/// Traps if `src`/`size` are not a valid encoding of a principal, or if
/// the principal is not a subnet.
fn ic0_replication_factor(&self, src: usize, size: usize, heap: &[u8])
-> HypervisorResult<u32>;

/// This system call indicates the cycle cost of an inter-canister call,
/// i.e., `ic0.call_perform`.
///
/// The cost is determined by the byte length of the method name and the
/// length of the encoded payload.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_call(
&self,
method_name_size: u64,
payload_size: u64,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()>;

/// This system call indicates the cycle cost of creating a canister on
/// the same subnet, i.e., the management canister's `create_canister`.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_create_canister(&self, dst: usize, heap: &mut [u8]) -> HypervisorResult<()>;

/// This system call indicates the cycle cost of making an http outcall,
/// i.e., the management canister's `http_request`.
///
/// `request_size` is the sum of the lengths of the variable request parts, as
/// documented in the interface specification.
/// `max_res_bytes` is the maximum number of response bytes the caller wishes to
/// accept.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_http_request(
&self,
request_size: u64,
max_res_bytes: u64,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()>;

/// This system call indicates the cycle cost of signing with ecdsa,
/// i.e., the management canister's `sign_with_ecdsa`, for the key
/// with name given by `src` + `size` and the provided curve.
///
/// Traps if `src`/`size` cannot be decoded to a valid key name.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_sign_with_ecdsa(
&self,
src: usize,
size: usize,
curve: EcdsaCurve,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()>;

/// This system call indicates the cycle cost of signing with schnorr,
/// i.e., the management canister's `sign_with_schnorr` for the key
/// with name given by `src` + `size` and the provided algorithm.
///
/// Traps if `src`/`size` cannot be decoded to a valid key name.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_sign_with_schnorr(
&self,
src: usize,
size: usize,
algorithm: SchnorrAlgorithm,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()>;

/// This system call indicates the cycle cost of threshold key derivation,
/// i.e., the management canister's `vetkd_derive_encrypted_key` for the key
/// with name given by `src` + `size` and the provided curve.
///
/// Traps if `src`/`size` cannot be decoded to a valid key name.
///
/// The amount of cycles is represented by a 128-bit value and is copied
/// to the canister memory starting at the location `dst`.
fn ic0_cost_vetkd_derive_encrypted_key(
&self,
src: usize,
size: usize,
curve: VetKdCurve,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()>;

/// Used to look up the size of the subnet Id of the calling canister.
fn ic0_subnet_self_size(&self) -> HypervisorResult<usize>;

Expand Down
11 changes: 11 additions & 0 deletions rs/interfaces/src/execution_environment/errors.rs
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,8 @@ pub enum HypervisorError {
bytes: NumBytes,
limit: NumBytes,
},
/// A user-specified Principal was not a valid subnet.
SubnetNotFound,
}

impl From<WasmInstrumentationError> for HypervisorError {
Expand All @@ -209,6 +211,9 @@ impl From<WasmEngineError> for HypervisorError {
impl std::fmt::Display for HypervisorError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::SubnetNotFound => {
write!(f, "User-specified Principal was not a valid subnet.")
}
Self::FunctionNotFound(table_idx, func_idx) => write!(
f,
"Canister requested to invoke a non-existent Wasm function {} from table {}",
Expand Down Expand Up @@ -401,6 +406,10 @@ impl AsErrorHelp for HypervisorError {
Self::FunctionNotFound(_, _)
| Self::ToolchainContractViolation { .. }
| Self::InvalidPrincipalId(_) => ErrorHelp::ToolchainError,
Self::SubnetNotFound => ErrorHelp::UserError {
suggestion: "Check the provided principal (not a subnet).".to_string(),
doc_link: doc_ref(""),
},
Self::MethodNotFound(_) => ErrorHelp::UserError {
suggestion: "Check that the method being called is exported by \
the target canister."
Expand Down Expand Up @@ -516,6 +525,7 @@ impl HypervisorError {
};

let code = match self {
Self::SubnetNotFound => E::SubnetNotFound,
Self::FunctionNotFound(_, _) => E::CanisterFunctionNotFound,
Self::MethodNotFound(_) => E::CanisterMethodNotFound,
Self::ToolchainContractViolation { .. } => E::CanisterContractViolation,
Expand Down Expand Up @@ -557,6 +567,7 @@ impl HypervisorError {
/// e.g. as a metric label.
pub fn as_str(&self) -> &'static str {
match self {
HypervisorError::SubnetNotFound => "SubnetNotFound",
HypervisorError::FunctionNotFound(..) => "FunctionNotFound",
HypervisorError::MethodNotFound(_) => "MethodNotFound",
HypervisorError::ToolchainContractViolation { .. } => "ToolchainContractViolation",
Expand Down
186 changes: 184 additions & 2 deletions rs/system_api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,14 @@ use ic_interfaces::execution_environment::{
TrapCode::{self, CyclesAmountTooBigFor64Bit},
};
use ic_logger::{error, ReplicaLogger};
use ic_management_canister_types::{
EcdsaCurve, EcdsaKeyId, MasterPublicKeyId, SchnorrAlgorithm, SchnorrKeyId, VetKdCurve,
VetKdKeyId,
};
use ic_registry_subnet_type::SubnetType;
use ic_replicated_state::{
canister_state::WASM_PAGE_SIZE_IN_BYTES, memory_required_to_push_request, Memory, NumWasmPages,
PageIndex,
canister_state::WASM_PAGE_SIZE_IN_BYTES, memory_required_to_push_request, Memory,
NetworkTopology, NumWasmPages, PageIndex,
};
use ic_sys::PageBytes;
use ic_types::{
Expand All @@ -36,6 +40,7 @@ use std::{
collections::BTreeMap,
convert::{From, TryFrom},
rc::Rc,
str,
};

pub mod cycles_balance_change;
Expand Down Expand Up @@ -3512,6 +3517,165 @@ impl SystemApi for SystemApiImpl {
result
}

fn ic0_replication_factor(
&self,
src: usize,
size: usize,
heap: &[u8],
) -> HypervisorResult<u32> {
let msg_bytes = valid_subslice("ic0_replication_factor", src, size, heap)?;
let subnet_id = PrincipalId::try_from(msg_bytes)
.map_err(|e| HypervisorError::InvalidPrincipalId(PrincipalIdBlobParseError(e.0)))?;
let result = self
.sandbox_safe_system_state
.network_topology
.get_subnet_size(&subnet_id.into())
.map(|x| x as u32)
.ok_or(HypervisorError::SubnetNotFound);
trace_syscall!(self, ReplicationFactor, result);
result
}

fn ic0_cost_call(
&self,
method_name_size: u64,
payload_size: u64,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()> {
let subnet_size = self.sandbox_safe_system_state.subnet_size;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.xnet_call_performed_fee(subnet_size)
+ self
.sandbox_safe_system_state
.cycles_account_manager
.xnet_call_bytes_transmitted_fee(
(method_name_size + payload_size).into(),
subnet_size,
);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_call")?;
trace_syscall!(self, CostCall, cost);
Ok(())
}

fn ic0_cost_create_canister(&self, dst: usize, heap: &mut [u8]) -> HypervisorResult<()> {
let subnet_size = self.sandbox_safe_system_state.subnet_size;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.canister_creation_fee(subnet_size);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_create_canister")?;
trace_syscall!(self, CostCreateCanister, cost);
Ok(())
}

fn ic0_cost_http_request(
&self,
request_size: u64,
max_res_bytes: u64,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()> {
let subnet_size = self.sandbox_safe_system_state.subnet_size;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.http_request_fee(request_size.into(), Some(max_res_bytes.into()), subnet_size);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_http_request")?;
trace_syscall!(self, CostHttpRequest, cost);
Ok(())
}

fn ic0_cost_sign_with_ecdsa(
&self,
src: usize,
size: usize,
curve: EcdsaCurve,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()> {
let key_bytes = valid_subslice("ic0.cost_sign_with_ecdsa heap", src, size, heap)?;
let name = str::from_utf8(key_bytes)
.map_err(|_| HypervisorError::ToolchainContractViolation {
error: format!(
"Failed to decode key name {}",
String::from_utf8_lossy(key_bytes)
),
})?
.to_string();
let key = MasterPublicKeyId::Ecdsa(EcdsaKeyId { curve, name });
let topology = &self.sandbox_safe_system_state.network_topology;
let subnet_size = get_key_replication_factor(topology, key)?;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.ecdsa_signature_fee(subnet_size);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_sign_with_ecdsa")?;
trace_syscall!(self, CostSignWithEcdsa, cost);
Ok(())
}

fn ic0_cost_sign_with_schnorr(
&self,
src: usize,
size: usize,
algorithm: SchnorrAlgorithm,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()> {
let key_bytes = valid_subslice("ic0.cost_sign_with_schnorr heap", src, size, heap)?;
let name = str::from_utf8(key_bytes)
.map_err(|_| HypervisorError::ToolchainContractViolation {
error: format!(
"Failed to decode key name {}",
String::from_utf8_lossy(key_bytes)
),
})?
.to_string();
let key = MasterPublicKeyId::Schnorr(SchnorrKeyId { algorithm, name });
let topology = &self.sandbox_safe_system_state.network_topology;
let subnet_size = get_key_replication_factor(topology, key)?;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.schnorr_signature_fee(subnet_size);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_sign_with_schnorr")?;
trace_syscall!(self, CostSignWithSchnorr, cost);
Ok(())
}

fn ic0_cost_vetkd_derive_encrypted_key(
&self,
src: usize,
size: usize,
curve: VetKdCurve,
dst: usize,
heap: &mut [u8],
) -> HypervisorResult<()> {
let key_bytes =
valid_subslice("ic0.cost_vetkd_derive_encrypted_key heap", src, size, heap)?;
let name = str::from_utf8(key_bytes)
.map_err(|_| HypervisorError::ToolchainContractViolation {
error: format!(
"Failed to decode key name {}",
String::from_utf8_lossy(key_bytes)
),
})?
.to_string();
let key = MasterPublicKeyId::VetKd(VetKdKeyId { curve, name });
let topology = &self.sandbox_safe_system_state.network_topology;
let subnet_size = get_key_replication_factor(topology, key)?;
let cost = self
.sandbox_safe_system_state
.cycles_account_manager
.vetkd_fee(subnet_size);
copy_cycles_to_heap(cost, dst, heap, "ic0_cost_vetkd_derive_encrypted_key")?;
trace_syscall!(self, CostVetkdDeriveEncryptedKey, cost);
Ok(())
}

fn ic0_subnet_self_size(&self) -> HypervisorResult<usize> {
let result = match &self.api_type {
ApiType::Start { .. } => Err(self.error_for("ic0_subnet_self_size")),
Expand Down Expand Up @@ -3575,6 +3739,24 @@ impl SystemApi for SystemApiImpl {
}
}

/// Look up key in `chain_key_enabled_subnets`, then extract all subnets
/// for that key and return the replication factor of the biggest one.
fn get_key_replication_factor(
topology: &NetworkTopology,
key: MasterPublicKeyId,
) -> HypervisorResult<usize> {
let subnets_with_key = topology.chain_key_enabled_subnets(&key);
subnets_with_key
.iter()
.map(|subnet_id| {
topology.get_subnet_size(subnet_id).unwrap() // we got the subnet_id from the same collection
})
.max()
.ok_or(HypervisorError::ToolchainContractViolation {
error: format!("Public key {:?} not known.", key),
})
}

/// The default implementation of the `OutOfInstructionHandler` trait.
/// It simply returns an out-of-instructions error.
#[derive(Default)]
Expand Down
Loading
Loading