From 464dd396980b5a28ca0c20daaa2fbc805743cc05 Mon Sep 17 00:00:00 2001 From: amela Date: Tue, 17 Oct 2023 11:04:36 +0200 Subject: [PATCH] docs/node/mainnet: Add 23.0 upgrade instructions for the Mainnet --- docs/node/mainnet/e-upgrade.md | 318 ++++++++++++++++++++++++++++++++- 1 file changed, 315 insertions(+), 3 deletions(-) diff --git a/docs/node/mainnet/e-upgrade.md b/docs/node/mainnet/e-upgrade.md index 04abb2d0a6..2fa3806f1a 100644 --- a/docs/node/mainnet/e-upgrade.md +++ b/docs/node/mainnet/e-upgrade.md @@ -6,7 +6,7 @@ This document is a work in progress and details around when the upgrade will be proposed and how it will be called are still not known and will be determined by the Oasis community. -The upgrade will first happen on Testnet and will only be proposed for Mainnet +The upgrade first happened on Testnet and will only be proposed for Mainnet after sufficient testing has been done to ensure everything will progress as planned. @@ -17,8 +17,8 @@ upgrade. :::caution -The E* upgrade on Mainnet is scheduled at epoch TODO which will happen -around TODO. +The E* upgrade on Mainnet is scheduled at epoch TBD which will happen +around TBD. ::: @@ -100,3 +100,315 @@ upgrade. For now, please take a look at the related [Testnet upgrade]. ::: [Testnet upgrade]: ../testnet/upgrade-log.md + +### Voting + +On TBD, an upgrade proposal will be proposed which (if accepted) will schedule +the upgrade on epoch **TBD**. + +For optimal voting experience, we recommend using the [Oasis CLI]. +Follow these steps to cast your vote: +1. [Import your keys into the wallet] +2. [Cast your vote] + +### Upgrade Instructions + +The following steps should be performed only after the network has reached the +upgrade epoch and has halted: + +1. Wait for the network to reach the upgrade epoch and halt. The node + will automatically stop without any action required on your part. After the + network has halted, proceed to the next steps. + +2. Download the Mainnet genesis file published in the + [Mainnet TBD release]. + +:::info + +Mainnet state at epoch **TBD** will be exported and migrated to a 23.0.x +compatible genesis file. + +The new genesis file will be published on the above link soon after reaching the +upgrade epoch. + +::: + +3. Verify the provided Mainnet upgrade genesis file by comparing it to the + local network state dump. + + Find the `genesis-mainnet-TBD.json` file in the `exports` subdirectory in + your data dir (e.g. `/node/`, `/srv/oasis/node/`) and run `sha256sum` on it. + Afterwards, compare it with the hash that we will share on the + `#node-operators` Discord channel. + + The state changes are described in the [State Changes](#state-changes) + section below. + +4. Replace the old genesis file with the new Mainnet genesis file. + +5. Ensure your node will remain stopped by disabling auto-starting via your + process manager (e.g., [systemd] or [Supervisor]). + +6. Back up the entire data directory of your node. Verify that the backup +includes the following folders: +- for consensus: `tendermint/abci-state` and `tendermint/data` +- for runtimes: `runtimes/*/mkvs_storage.badger.db` and +`runtimes/*/worker-local-storage.badger.db` + +7. [Wipe state]. This must be performed _before_ replacing the Oasis Node + binary. In case you are upgrading ParaTimes/runtimes ensure you read the + following section: + +:::danger + +State of ParaTimes/runtimes is not affected by this upgrade and MUST NOT be +wiped. Wiping state for confidential ParaTimes will prevent your compute or +key manager node from transitioning to the new network. + +To safely wipe the blockchain state on a runtime while preserving the runtime +state, follow these steps: +1. **Dry run:** initiate a dry run to preview which files will be deleted by +running the following command: +``` bash +# specify 'datadir' as your node's data directory +oasis-node unsafe-reset \ + --datadir=/serverdir/node \ + --dry_run +``` +2. **Wipe blockchain state:** after reviewing the dry run results, proceed with +the reset by running: +``` bash +# specify 'datadir' as your node's data directory +oasis-node unsafe-reset \ + --datadir=/serverdir/node +``` + +Transitioning confidential ParaTimes to the new network requires local state +that is sealed to the CPU. This also means that bootstrapping a new node on a +separate CPU immediately after the network upgrade will not be possible until +an updated ParaTime containing new trust roots is released and adopted. + +::: + +8. Replace the old version of Oasis Node with version [23.0.x]. + +:::info + +The Oasis Core 23.0.x binary in our published releases is built only for Ubuntu +22.04 (GLIBC>=2.32). You'll have to build it yourself if you're using prior +Ubuntu versions (or other distributions using older system libraries). + +::: + +9. Perform any needed [configuration changes](#configuration-changes) described + below. + +10. (only Rosetta Gateway operators) Replace old version of Oasis Rosetta + Gateway with version [2.6.0][rosetta-gw-2.6.0]. + +11. Start your node and re-enable auto-starting via your process manager. + +[Oasis CLI]: ../../general/manage-tokens/cli/ +[Import your keys into the wallet]: ../../general/manage-tokens/cli/wallet.md#import-file +[Cast your vote]: ../../general/manage-tokens/cli/network.md#governance-cast-vote +[Mainnet TBD release]: https://github.com/oasisprotocol/mainnet-artifacts/releases/tag/TBD +[systemd]: https://systemd.io/ +[Supervisor]: http://supervisord.org/ +[Wipe state]: ../run-your-node/maintenance/wiping-node-state.md#state-wipe-and-keep-node-identity +[23.0.x]: https://github.com/oasisprotocol/oasis-core/releases/tag/v23.0.x +[rosetta-gw-2.6.0]: + https://github.com/oasisprotocol/oasis-rosetta-gateway/releases/tag/v2.6.0 + +### Configuration Changes + +:::info + +To see the full extent of the changes examine the [Change Log] of the 23.0.x +release. + +::: + +The node configuration has been refactored so that everything is now configured +via a YAML configuration file and **configuring via command-line options is no +longer supported**. + +Some configuration options have changed and so the configuration file needs to +be updated. To make this step easier, a command-line tool has been provided that +will perform most of the changes automatically. You can run it with: + +``` +oasis-node config migrate --in config.yml --out new-config.yml +``` + +The migration subcommand logs the various changes it makes and warns you if a +config option is no longer supported, etc. At the end, any unknown sections of +the input config file are printed to the terminal to give you a chance to review +them and make manual changes if required. + +Note that the migration subcommand does not preserve comments and order of +sections from the input YAML config file. You should always carefully read the +output of this command, as well as compare the generated config file with the +original before using it. + +After you are satisfied with the new configuration file, replace the old file +with the new one as follows: + +``` +mv new-config.yml config.yml +``` + +:::tip + +The configuration format for seed nodes has changed and it now requires the +node's P2P public key to be used. In case your old configuration file contains +known Mainnet seed nodes, this transformation is performed automatically. + +However, if it contains unknown seed nodes then the conversion did not happen +automatically and you may need to obtain the seed node's P2P public key. For +Mainnet you can use the following addresses: + +* `TBD` +* `TBD` + +Please be aware that every seed node should be configured to listen on two +distinct ports. One is dedicated to peer discovery within the CometBFT P2P +network, while the other is used to bootstrap the Oasis P2P network. + +::: + +[Change Log]: + https://github.com/oasisprotocol/oasis-core/blob/v23.0.x/CHANGELOG.md + +### Data Directory Changes + +The subdirectory (located inside the node's data directory) used to store +consensus-related data, previously called `tendermint` (after the consensus +layer protocol backend) has been renamed to `consensus` in Oasis Core 23.0.x. If +any of your scripts rely on specific directory names, please make sure to update +them to reflect the changed sdirectory name. + +### State Changes + +The following parts of the genesis document will be updated: + +:::info + +For a more detailed explanation of the parameters below, see the +[Genesis Document] docs. + +::: + +:::info + +All state changes will be done automatically with the migration command provided +by the new version of `oasis-node`. It can be used as follows to derive the same +genesis file from an existing state dump at the correct height (assuming there +is a `genesis.json` present in the current working directory): + +``` +oasis-node genesis migrate --genesis.new_chain_id oasis-4 +``` + +::: + +#### General + +* **`chain_id`** will be set to `oasis-4`. + +* **`halt_epoch`** will be removed as it is no longer used. + +#### Registry + +* **`registry.runtimes[].txn_scheduler.propose_batch_timeout`** specifies how + long to wait before accepting proposal from the next backup scheduler. It will + be set to `5000000000` (5 seconds). Previously the value was represented in + the number of consensus layer blocks. + +* **`registry.params.gas_costs.prove_freshness`** specifies the cost of the + freshness proof transaction. It will be set to `1000`. + +* **`registry.params.gas_costs.update_keymanager`** specifies the cost of the + keymanager policy update transaction. It will be removed as the parameter has + been moved under `keymanager.params.gas_costs.update_policy`. + +* **`registry.params.tee_features`** specify various TEE features supported by + the consensus layer registry service. These will be set to the following + values to activate the new features: + + ```json + "tee_features": { + "sgx": { + "pcs": true, + "signed_attestations": true, + "max_attestation_age": 1200 + }, + "freshness_proofs": true + } + ``` + +* **`registry.params.max_runtime_deployments`** specifies the maximum number of + runtime deployments that can be specified in the runtime descriptor. It will + be set to `5`. + +#### Root Hash + +* **`roothash.params.max_past_roots_stored`** specifies the maximum number of + past runtime state roots that are stored in consensus state for each runtime. + It will be set to `1200`. + +#### Staking + +* **`staking.params.commission_schedule_rules.min_commission_rate`** specifies + the minimum commission rate. It will be set to `0` to maintain the existing + behavior. + +* **`staking.params.thresholds.node-observer`** specifies the stake threshold + for registering an observer node. It will be set to `100000000000` base units + (or `100` tokens), same as for existing compute nodes. + +#### Key Manager + +* **`keymanager.params.gas_costs`** specify the cost of key manager + transactions. These will be set to the following values: + + ```json + "gas_costs": { + "publish_ephemeral_secret": 1000, + "publish_master_secret": 1000, + "update_policy": 1000 + } + ``` + +#### Random Beacon + +* **`beacon.base`** is the network's starting epoch. It will be set to the epoch + of Mainnet's state dump + 1, `TBD`. + +#### Governance + +* **`governance.params.enable_change_parameters_proposal`** specifies whether + parameter change governance proposals are allowed. It will be set to `true`. + +#### Consensus + +* **`consensus.params.max_block_size`** specifies the maximum block size in the + consensus layer. It will be set to `1048576` (1 MiB). + +#### Other + +* **`extra_data`** will be set back to the value in the [Mainnet genesis file] + to include the Oasis Network's genesis quote: + + _”_[_Quis custodiet ipsos custodes?_][mainnet-quote]_” \[submitted by Oasis + Community Member Daniyar Borangaziyev]:_ + + ``` + "extra_data": { + "quote": "UXVpcyBjdXN0b2RpZXQgaXBzb3MgY3VzdG9kZXM/IFtzdWJtaXR0ZWQgYnkgT2FzaXMgQ29tbXVuaXR5IE1lbWJlciBEYW5peWFyIEJvcmFuZ2F6aXlldl0=" + } + ``` + +[Genesis Document]: ../genesis-doc.md#parameters +[Mainnet genesis file]: + https://github.com/oasisprotocol/mainnet-artifacts/releases/tag/2020-11-18 +[mainnet-quote]: https://en.wikipedia.org/wiki/Quis_custodiet_ipsos_custodes%3F