Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/polkadot sdk v1.16.0 #827

Merged
merged 105 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
105 commits
Select commit Hold shift + click to select a range
1f3851f
Merge pull request #758 from opentensor/devnet
unconst Aug 27, 2024
aaf9149
Merge pull request #767 from opentensor/devnet-ready
unconst Aug 28, 2024
97b11de
Merge pull request #769 from opentensor/devnet
unconst Aug 28, 2024
24d5819
feat: use nucleus archive node
Aug 29, 2024
460f175
Merge pull request #773 from opentensor/fix/nucleus_archive_node
unconst Aug 29, 2024
e79b099
Merge pull request #770 from opentensor/testnet
unconst Aug 30, 2024
cf75652
feat: bumps testnet total issuance
Sep 4, 2024
bcf28bd
Merge pull request #776 from opentensor/feat/bump_testnet_total_issuance
unconst Sep 4, 2024
ffd8ed3
Revert "hotfix: bumps testnet total issuance"
distributedstatemachine Sep 5, 2024
ca42a0f
fix: metagraph stake info
Sep 5, 2024
8dd149c
chore: bump spec version
Sep 5, 2024
dbb5b5d
Merge pull request #781 from opentensor/hotfix/metagraph_stake
unconst Sep 5, 2024
94256d5
Merge pull request #783 from opentensor/devnet
distributedstatemachine Sep 5, 2024
b1039b2
Merge pull request #779 from opentensor/revert-776-feat/bump_testnet_…
unconst Sep 5, 2024
58c1f51
Merge pull request #785 from opentensor/testnet
unconst Sep 6, 2024
3412916
hotfit: dont drain root emisions
Sep 9, 2024
cf768eb
filter 0
Sep 9, 2024
adad4fd
Merge pull request #790 from opentensor/hotfix/root_emissions
unconst Sep 9, 2024
6c8d794
reorder mul/div
camfairchild Sep 12, 2024
ac5bc9a
add test for overflow
camfairchild Sep 12, 2024
28df894
bump spec ver
camfairchild Sep 12, 2024
c74baec
re-apply hotfix patch
camfairchild Sep 12, 2024
faf9aca
bump spec ver
camfairchild Sep 12, 2024
cdf33a8
update test for temp fix
camfairchild Sep 12, 2024
824c5f5
update build script for wasm building
camfairchild Sep 12, 2024
abd87c6
Merge pull request #808 from opentensor/hotfix/fix-emission-calc
unconst Sep 12, 2024
a7a94c4
fix swap hotkey func: needs test
camfairchild Sep 10, 2024
9969452
add migrations for already-swapped keys
camfairchild Sep 10, 2024
ef2457d
also fix swap coldkey: needs test
camfairchild Sep 10, 2024
dcd06f1
add test for coldkey swap
camfairchild Sep 10, 2024
dfd96f6
add tests for swap hotkey
camfairchild Sep 10, 2024
12c4d1f
remove increase stake swap migrate
camfairchild Sep 11, 2024
6a2eeb6
make sure to keep old hotkey pending emissions
camfairchild Sep 11, 2024
278eb87
tests for new migration
camfairchild Sep 11, 2024
ddb5184
use same function for both key-pairs
camfairchild Sep 11, 2024
edc86b1
match on results instead
camfairchild Sep 11, 2024
87a5e3c
chore: appease clippy
camfairchild Sep 11, 2024
b07fdbb
Hotfix/only exclude add stake from emission no packing (#807)
camfairchild Sep 12, 2024
ea7e1de
bump spec
camfairchild Sep 12, 2024
f2b367c
update secret to var
camfairchild Sep 12, 2024
475888f
remove test with LastAddStakeIncrease
camfairchild Sep 12, 2024
4f3f151
Merge pull request #806 from opentensor/hotfix/patch-new-tempo-upgrade
unconst Sep 12, 2024
1a5009a
add migration patch; remove pending emission from old hotkey
camfairchild Sep 13, 2024
d1c25d3
check stake after migration
camfairchild Sep 13, 2024
eea273a
spec version
camfairchild Sep 13, 2024
16d636a
use check helper in set weights call
camfairchild Sep 13, 2024
eea8d1a
add test for set weights checks on child keys
camfairchild Sep 13, 2024
a0b87da
add test for regular keys to prevent regression
camfairchild Sep 13, 2024
fff2529
add test for total issuance/stake counters
camfairchild Sep 13, 2024
773670c
address tests
camfairchild Sep 13, 2024
9f4b90c
add new migration method and more testing
camfairchild Sep 13, 2024
870689f
chore: clippy
camfairchild Sep 13, 2024
60fd528
spec version only 1 more than mainnet
camfairchild Sep 13, 2024
4ad2c5b
use new migration multisig
camfairchild Sep 13, 2024
625fc9c
remove old migration runner
Sep 16, 2024
67aad02
add try runtime check for new migration
Sep 16, 2024
de41a34
make check idempot
Sep 17, 2024
c421589
chore: clippy
Sep 17, 2024
4c2629e
Add back old migration and fix var names
Sep 17, 2024
af585b9
Merge pull request #809 from opentensor/hotfix/pending-emission-migra…
unconst Sep 17, 2024
af79b30
Update to Polkadot SDK 1.16.0-rc1, wip: benchmarks don't work
gztensor Sep 19, 2024
5611770
Add token argument to the publish script
keithtensor Aug 29, 2024
d9e7c5e
Rebase onto devnet-ready
gztensor Sep 20, 2024
d0ce4d2
simplify cargo test CI step, always check benchmarks + other feats
sam0x17 Aug 27, 2024
9fe01d5
add root weights proxy
camfairchild Aug 27, 2024
8a1683f
fmt
camfairchild Aug 27, 2024
6840702
fix typo
camfairchild Aug 27, 2024
d6e5f96
Rebase onto devnet-ready
gztensor Sep 20, 2024
db41648
import original script
sam0x17 Sep 9, 2024
1ffd1fd
scaffold
sam0x17 Sep 9, 2024
d837a6f
parsing of previous tag
sam0x17 Sep 9, 2024
28c1ede
parse network and all tags
sam0x17 Sep 9, 2024
9323877
resolve previous tag
sam0x17 Sep 9, 2024
9daafe1
narrow down to filtered merges
sam0x17 Sep 9, 2024
c9bfdd2
branch selection, fallible eval method
sam0x17 Sep 9, 2024
550f547
tweak
sam0x17 Sep 9, 2024
b57dafe
WIP
sam0x17 Sep 9, 2024
9219693
get PR titles
sam0x17 Sep 10, 2024
f00159a
resolve PR titles properly
sam0x17 Sep 10, 2024
a918435
authors working :tada:
sam0x17 Sep 10, 2024
e81e5ea
working
sam0x17 Sep 10, 2024
26c8d19
write release notes to /tmp/release_notes.md
sam0x17 Sep 10, 2024
322618c
automatically re-run deployment check when labels changed
sam0x17 Sep 10, 2024
9b28037
tweak
sam0x17 Sep 10, 2024
40323e8
debug
sam0x17 Sep 10, 2024
ff84f27
bump CI
sam0x17 Sep 10, 2024
dce4908
whoops
sam0x17 Sep 10, 2024
4f6113d
fix conditional
sam0x17 Sep 10, 2024
5c1d194
remove label debug step
sam0x17 Sep 10, 2024
75d5c16
tweak triggers
sam0x17 Sep 10, 2024
da7054f
check on-push
sam0x17 Sep 10, 2024
0686dd2
fix
sam0x17 Sep 10, 2024
f505381
remove merged_script.sh
sam0x17 Sep 10, 2024
9b233cb
fix workspace lint error
sam0x17 Sep 10, 2024
5ba62d0
Update to Polkadot SDK 1.16.0-rc1, wip: benchmarks don't work
gztensor Sep 19, 2024
457b497
Format
gztensor Sep 20, 2024
001bec2
Remove unused import in tests
gztensor Sep 20, 2024
4cbbbf4
Merge branch 'devnet-ready' into feat/polkadot-sdk-v1.16.0
gztensor Sep 20, 2024
bf5e375
Revert changes pulled from main
gztensor Sep 21, 2024
a388875
Add a new line to check finney workflow
gztensor Sep 21, 2024
b1266d6
Cleanup newlines
gztensor Sep 21, 2024
dfe987a
clippy
gztensor Sep 21, 2024
ce5fdf1
Merge devnet-ready into feat/polkadot-sdk-v1.16.0
gztensor Sep 24, 2024
00f6a16
Add rpc-external flag to localhost.sh for e2e tests
gztensor Sep 24, 2024
85a506a
Bump version to pass deploy check. Use --unsafe-force-node-key-genera…
gztensor Sep 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3,039 changes: 1,858 additions & 1,181 deletions Cargo.lock

Large diffs are not rendered by default.

142 changes: 71 additions & 71 deletions Cargo.toml

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions node/src/chain_spec/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ pub mod finney;
pub mod localnet;
pub mod testnet;

use node_subtensor_runtime::{AccountId, Block, RuntimeGenesisConfig, Signature, WASM_BINARY};
use node_subtensor_runtime::{AccountId, Block, Signature, WASM_BINARY};
use sc_chain_spec_derive::ChainSpecExtension;
use sc_service::ChainType;
use sp_consensus_aura::sr25519::AuthorityId as AuraId;
Expand All @@ -32,7 +32,7 @@ pub struct Extensions {
}

/// Specialized `ChainSpec`. This is a specialization of the general Substrate ChainSpec type.
pub type ChainSpec = sc_service::GenericChainSpec<RuntimeGenesisConfig, Extensions>;
pub type ChainSpec = sc_service::GenericChainSpec<Extensions>;

/// Generate a crypto pair from seed.
pub fn get_from_seed<TPublic: Public>(seed: &str) -> <TPublic::Pair as Pair>::Public {
Expand Down
63 changes: 41 additions & 22 deletions node/src/command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,13 @@ pub use frame_benchmarking_cli::{BenchmarkCmd, ExtrinsicFactory, SUBSTRATE_REFER
pub use node_subtensor_runtime::EXISTENTIAL_DEPOSIT;
#[cfg(feature = "runtime-benchmarks")]
pub use sp_keyring::Sr25519Keyring;
#[cfg(feature = "runtime-benchmarks")]
use sp_runtime::traits::HashingFor;

use node_subtensor_runtime::Block;
use sc_cli::SubstrateCli;
use sc_service::{Configuration, PartialComponents};
use sc_service::{
config::{ExecutorConfiguration, RpcConfiguration},
Configuration, PartialComponents,
};

impl SubstrateCli for Cli {
fn impl_name() -> String {
Expand Down Expand Up @@ -149,7 +150,9 @@ pub fn run() -> sc_cli::Result<()> {
);
}

cmd.run::<HashingFor<Block>, service::ExecutorDispatch>(config)
cmd.run_with_spec::<sp_runtime::traits::HashingFor<Block>, ()>(Some(
config.chain_spec,
))
}
BenchmarkCmd::Block(cmd) => {
let PartialComponents { client, .. } = service::new_partial(&config)?;
Expand Down Expand Up @@ -210,7 +213,18 @@ pub fn run() -> sc_cli::Result<()> {
let runner = cli.create_runner(&cli.run)?;
runner.run_node_until_exit(|config| async move {
let config = override_default_heap_pages(config, 60_000);
service::new_full(config).map_err(sc_cli::Error::Service)
match config.network.network_backend {
sc_network::config::NetworkBackendType::Libp2p => service::new_full::<
sc_network::NetworkWorker<
node_subtensor_runtime::opaque::Block,
<node_subtensor_runtime::opaque::Block as sp_runtime::traits::Block>::Hash,
>,
>(config)
.map_err(sc_cli::Error::Service),
sc_network::config::NetworkBackendType::Litep2p =>
service::new_full::<sc_network::Litep2pNetworkBackend>(config)
.map_err(sc_cli::Error::Service),
}
})
}
}
Expand All @@ -219,7 +233,6 @@ pub fn run() -> sc_cli::Result<()> {
/// Override default heap pages
fn override_default_heap_pages(config: Configuration, pages: u64) -> Configuration {
Configuration {
default_heap_pages: Some(pages),
impl_name: config.impl_name,
impl_version: config.impl_version,
role: config.role,
Expand All @@ -232,20 +245,7 @@ fn override_default_heap_pages(config: Configuration, pages: u64) -> Configurati
state_pruning: config.state_pruning,
blocks_pruning: config.blocks_pruning,
chain_spec: config.chain_spec,
wasm_method: config.wasm_method,
wasm_runtime_overrides: config.wasm_runtime_overrides,
rpc_addr: config.rpc_addr,
rpc_max_connections: config.rpc_max_connections,
rpc_cors: config.rpc_cors,
rpc_methods: config.rpc_methods,
rpc_max_request_size: config.rpc_max_request_size,
rpc_max_response_size: config.rpc_max_response_size,
rpc_id_provider: config.rpc_id_provider,
rpc_max_subs_per_conn: config.rpc_max_subs_per_conn,
rpc_port: config.rpc_port,
rpc_message_buffer_capacity: config.rpc_message_buffer_capacity,
rpc_batch_config: config.rpc_batch_config,
rpc_rate_limit: config.rpc_rate_limit,
prometheus_config: config.prometheus_config,
telemetry_endpoints: config.telemetry_endpoints,
offchain_worker: config.offchain_worker,
Expand All @@ -254,11 +254,30 @@ fn override_default_heap_pages(config: Configuration, pages: u64) -> Configurati
dev_key_seed: config.dev_key_seed,
tracing_targets: config.tracing_targets,
tracing_receiver: config.tracing_receiver,
max_runtime_instances: config.max_runtime_instances,
announce_block: config.announce_block,
data_path: config.data_path,
base_path: config.base_path,
informant_output_format: config.informant_output_format,
runtime_cache_size: config.runtime_cache_size,
executor: ExecutorConfiguration {
default_heap_pages: Some(pages),
wasm_method: config.executor.wasm_method,
max_runtime_instances: config.executor.max_runtime_instances,
runtime_cache_size: config.executor.runtime_cache_size,
},
rpc: RpcConfiguration {
addr: config.rpc.addr,
max_connections: config.rpc.max_connections,
cors: config.rpc.cors,
methods: config.rpc.methods,
max_request_size: config.rpc.max_request_size,
max_response_size: config.rpc.max_response_size,
id_provider: config.rpc.id_provider,
max_subs_per_conn: config.rpc.max_subs_per_conn,
port: config.rpc.port,
message_buffer_capacity: config.rpc.message_buffer_capacity,
batch_config: config.rpc.batch_config,
rate_limit: config.rpc.rate_limit,
rate_limit_whitelisted_ips: config.rpc.rate_limit_whitelisted_ips,
rate_limit_trust_proxy_headers: config.rpc.rate_limit_trust_proxy_headers,
},
}
}
7 changes: 1 addition & 6 deletions node/src/rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ use sp_api::ProvideRuntimeApi;
use sp_block_builder::BlockBuilder;
use sp_blockchain::{Error as BlockChainError, HeaderBackend, HeaderMetadata};

pub use sc_rpc_api::DenyUnsafe;

/// Dependencies for GRANDPA
pub struct GrandpaDeps<B> {
/// Voting round info.
Expand All @@ -37,8 +35,6 @@ pub struct FullDeps<C, P, B> {
pub client: Arc<C>,
/// Transaction pool instance.
pub pool: Arc<P>,
/// Whether to deny unsafe calls
pub deny_unsafe: DenyUnsafe,
/// Grandpa block import setup.
pub grandpa: GrandpaDeps<B>,
/// Backend used by the node.
Expand Down Expand Up @@ -72,15 +68,14 @@ where
let FullDeps {
client,
pool,
deny_unsafe,
grandpa,
_backend: _,
} = deps;

// Custom RPC methods for Paratensor
module.merge(SubtensorCustom::new(client.clone()).into_rpc())?;

module.merge(System::new(client.clone(), pool.clone(), deny_unsafe).into_rpc())?;
module.merge(System::new(client.clone(), pool.clone()).into_rpc())?;
module.merge(TransactionPayment::new(client).into_rpc())?;

let GrandpaDeps {
Expand Down
73 changes: 26 additions & 47 deletions node/src/service.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ use sc_client_api::{Backend, BlockBackend};
use sc_consensus_aura::{ImportQueueParams, SlotProportion, StartAuraParams};
use sc_consensus_grandpa::SharedVoterState;
use sc_consensus_slots::BackoffAuthoringOnFinalizedHeadLagging;
use sc_executor::sp_wasm_interface::{Function, HostFunctionRegistry, HostFunctions};
pub use sc_executor::NativeElseWasmExecutor;
use sc_service::{error::Error as ServiceError, Configuration, TaskManager, WarpSyncParams};
pub use sc_executor::WasmExecutor;
use sc_service::{error::Error as ServiceError, Configuration, TaskManager, WarpSyncConfig};
use sc_telemetry::{Telemetry, TelemetryWorker};
use sc_transaction_pool_api::OffchainTransactionPoolFactory;
use sp_consensus_aura::sr25519::AuthorityPair as AuraPair;
Expand All @@ -18,42 +17,8 @@ use std::{sync::Arc, time::Duration};
/// imported and generated.
const GRANDPA_JUSTIFICATION_PERIOD: u32 = 512;

// Our native executor instance.
pub struct ExecutorDispatch;

// appeasing the compiler, this is a no-op
impl HostFunctions for ExecutorDispatch {
fn host_functions() -> Vec<&'static dyn Function> {
vec![]
}

fn register_static<T>(_registry: &mut T) -> core::result::Result<(), T::Error>
where
T: HostFunctionRegistry,
{
Ok(())
}
}

impl sc_executor::NativeExecutionDispatch for ExecutorDispatch {
// Always enable runtime benchmark host functions, the genesis state
// was built with them so we're stuck with them forever.
//
// They're just a noop, never actually get used if the runtime was not compiled with
// `runtime-benchmarks`.
type ExtendHostFunctions = frame_benchmarking::benchmarking::HostFunctions;

fn dispatch(method: &str, data: &[u8]) -> Option<Vec<u8>> {
node_subtensor_runtime::api::dispatch(method, data)
}

fn native_version() -> sc_executor::NativeVersion {
node_subtensor_runtime::native_version()
}
}

pub(crate) type FullClient =
sc_service::TFullClient<Block, RuntimeApi, NativeElseWasmExecutor<ExecutorDispatch>>;
sc_service::TFullClient<Block, RuntimeApi, WasmExecutor<sp_io::SubstrateHostFunctions>>;
type FullBackend = sc_service::TFullBackend<Block>;
type FullSelectChain = sc_consensus::LongestChain<FullBackend, Block>;

Expand Down Expand Up @@ -90,7 +55,7 @@ pub fn new_partial(
})
.transpose()?;

let executor = sc_service::new_native_or_wasm_executor(config);
let executor = sc_service::new_wasm_executor::<sp_io::SubstrateHostFunctions>(&config.executor);

let (client, backend, keystore_container, task_manager) =
sc_service::new_full_parts::<Block, RuntimeApi, _>(
Expand Down Expand Up @@ -163,7 +128,11 @@ pub fn new_partial(
}

// Builds a new service for a full client.
pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
pub fn new_full<
N: sc_network::NetworkBackend<Block, <Block as sp_runtime::traits::Block>::Hash>,
>(
config: Configuration,
) -> Result<TaskManager, ServiceError> {
let sc_service::PartialComponents {
client,
backend,
Expand All @@ -175,7 +144,12 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
other: (block_import, grandpa_link, mut telemetry),
} = new_partial(&config)?;

let mut net_config = sc_network::config::FullNetworkConfiguration::new(&config.network);
let mut net_config = sc_network::config::FullNetworkConfiguration::<
Block,
<Block as sp_runtime::traits::Block>::Hash,
N,
>::new(&config.network, config.prometheus_registry().cloned());
let metrics = N::register_notification_metrics(config.prometheus_registry());

let grandpa_protocol_name = sc_consensus_grandpa::protocol_standard_name(
&client
Expand All @@ -186,8 +160,13 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
&config.chain_spec,
);

let peer_store_handle = net_config.peer_store_handle();
let (grandpa_protocol_config, grandpa_notification_service) =
sc_consensus_grandpa::grandpa_peers_set_config(grandpa_protocol_name.clone());
sc_consensus_grandpa::grandpa_peers_set_config::<_, N>(
grandpa_protocol_name.clone(),
metrics.clone(),
peer_store_handle,
);
net_config.add_notification_protocol(grandpa_protocol_config);

let warp_sync = Arc::new(sc_consensus_grandpa::warp_proof::NetworkProvider::new(
Expand All @@ -205,8 +184,9 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
spawn_handle: task_manager.spawn_handle(),
import_queue,
block_announce_validator_builder: None,
warp_sync_params: Some(WarpSyncParams::WithProvider(warp_sync)),
warp_sync_config: Some(WarpSyncConfig::WithProvider(warp_sync)),
block_relay: None,
metrics,
})?;

if config.offchain_worker.enabled {
Expand All @@ -221,7 +201,7 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
transaction_pool: Some(OffchainTransactionPoolFactory::new(
transaction_pool.clone(),
)),
network_provider: network.clone(),
network_provider: Arc::new(network.clone()),
enable_http_requests: true,
custom_extensions: |_| vec![],
})
Expand All @@ -239,7 +219,7 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
let shared_authority_set = grandpa_link.shared_authority_set().clone();
let shared_voter_state = SharedVoterState::empty();

let role = config.role.clone();
let role = config.role;
let force_authoring = config.force_authoring;
let backoff_authoring_blocks = Some(BackoffAuthoringOnFinalizedHeadLagging {
unfinalized_slack: 6,
Expand All @@ -254,11 +234,10 @@ pub fn new_full(config: Configuration) -> Result<TaskManager, ServiceError> {
let pool = transaction_pool.clone();

Box::new(
move |deny_unsafe, subscription_executor: sc_rpc::SubscriptionTaskExecutor| {
move |subscription_executor: sc_rpc::SubscriptionTaskExecutor| {
let deps = crate::rpc::FullDeps {
client: client.clone(),
pool: pool.clone(),
deny_unsafe,
grandpa: crate::rpc::GrandpaDeps {
shared_voter_state: shared_voter_state.clone(),
shared_authority_set: shared_authority_set.clone(),
Expand Down
1 change: 0 additions & 1 deletion pallets/commitments/src/tests.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#![allow(non_camel_case_types)]

use super::*;
use crate as pallet_commitments;
use frame_support::derive_impl;
use frame_support::traits::ConstU64;
Expand Down
Loading
Loading