Skip to content

Commit

Permalink
Add validator-manager (#3502)
Browse files Browse the repository at this point in the history
## Issue Addressed

Addresses #2557

## Proposed Changes

Adds the `lighthouse validator-manager` command, which provides:

- `lighthouse validator-manager create`
    - Creates a `validators.json` file and a `deposits.json` (same format as https://github.com/ethereum/staking-deposit-cli)
- `lighthouse validator-manager import`
    - Imports validators from a `validators.json` file to the VC via the HTTP API.
- `lighthouse validator-manager move`
    - Moves validators from one VC to the other, utilizing only the VC API.

## Additional Info

In 98bcb94 I've reduced some VC `ERRO` and `CRIT` warnings to `WARN` or `DEBG` for the case where a pubkey is missing from the validator store. These were being triggered when we removed a validator but still had it in caches. It seems to me that `UnknownPubkey` will only happen in the case where we've removed a validator, so downgrading the logs is prudent. All the logs are `DEBG` apart from attestations and blocks which are `WARN`. I thought having *some* logging about this condition might help us down the track.

In 856cd7e I've made the VC delete the corresponding password file when it's deleting a keystore. This seemed like nice hygiene. Notably, it'll only delete that password file after it scans the validator definitions and finds that no other validator is also using that password file.
  • Loading branch information
paulhauner committed Aug 8, 2023
1 parent 5ea7505 commit 1373dcf
Show file tree
Hide file tree
Showing 69 changed files with 6,058 additions and 743 deletions.
30 changes: 30 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,8 @@ members = [
"validator_client",
"validator_client/slashing_protection",

"validator_manager",

"watch",
]
resolver = "2"
Expand Down
2 changes: 2 additions & 0 deletions account_manager/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ safe_arith = {path = "../consensus/safe_arith"}
slot_clock = { path = "../common/slot_clock" }
filesystem = { path = "../common/filesystem" }
sensitive_url = { path = "../common/sensitive_url" }
serde = { version = "1.0.116", features = ["derive"] }
serde_json = "1.0.58"

[dev-dependencies]
tempfile = "3.1.0"
50 changes: 1 addition & 49 deletions account_manager/src/common.rs
Original file line number Diff line number Diff line change
@@ -1,55 +1,7 @@
use account_utils::PlainText;
use account_utils::{read_input_from_user, strip_off_newlines};
use eth2_wallet::bip39::{Language, Mnemonic};
use std::fs;
use std::path::PathBuf;
use std::str::from_utf8;
use std::thread::sleep;
use std::time::Duration;
use account_utils::read_input_from_user;

pub const MNEMONIC_PROMPT: &str = "Enter the mnemonic phrase:";
pub const WALLET_NAME_PROMPT: &str = "Enter wallet name:";

pub fn read_mnemonic_from_cli(
mnemonic_path: Option<PathBuf>,
stdin_inputs: bool,
) -> Result<Mnemonic, String> {
let mnemonic = match mnemonic_path {
Some(path) => fs::read(&path)
.map_err(|e| format!("Unable to read {:?}: {:?}", path, e))
.and_then(|bytes| {
let bytes_no_newlines: PlainText = strip_off_newlines(bytes).into();
let phrase = from_utf8(bytes_no_newlines.as_ref())
.map_err(|e| format!("Unable to derive mnemonic: {:?}", e))?;
Mnemonic::from_phrase(phrase, Language::English).map_err(|e| {
format!(
"Unable to derive mnemonic from string {:?}: {:?}",
phrase, e
)
})
})?,
None => loop {
eprintln!();
eprintln!("{}", MNEMONIC_PROMPT);

let mnemonic = read_input_from_user(stdin_inputs)?;

match Mnemonic::from_phrase(mnemonic.as_str(), Language::English) {
Ok(mnemonic_m) => {
eprintln!("Valid mnemonic provided.");
eprintln!();
sleep(Duration::from_secs(1));
break mnemonic_m;
}
Err(_) => {
eprintln!("Invalid mnemonic");
}
}
},
};
Ok(mnemonic)
}

/// Reads in a wallet name from the user. If the `--wallet-name` flag is provided, use it. Otherwise
/// read from an interactive prompt using tty unless the `--stdin-inputs` flag is provided.
pub fn read_wallet_name_from_cli(
Expand Down
8 changes: 5 additions & 3 deletions account_manager/src/validator/import.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ use account_utils::{
eth2_keystore::Keystore,
read_password_from_user,
validator_definitions::{
recursively_find_voting_keystores, ValidatorDefinition, ValidatorDefinitions,
CONFIG_FILENAME,
recursively_find_voting_keystores, PasswordStorage, ValidatorDefinition,
ValidatorDefinitions, CONFIG_FILENAME,
},
ZeroizeString,
};
Expand Down Expand Up @@ -277,7 +277,9 @@ pub fn cli_run(matches: &ArgMatches, validator_dir: PathBuf) -> Result<(), Strin
let suggested_fee_recipient = None;
let validator_def = ValidatorDefinition::new_keystore_with_password(
&dest_keystore,
password_opt,
password_opt
.map(PasswordStorage::ValidatorDefinitions)
.unwrap_or(PasswordStorage::None),
graffiti,
suggested_fee_recipient,
None,
Expand Down
3 changes: 1 addition & 2 deletions account_manager/src/validator/recover.rs
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
use super::create::STORE_WITHDRAW_FLAG;
use crate::common::read_mnemonic_from_cli;
use crate::validator::create::COUNT_FLAG;
use crate::wallet::create::STDIN_INPUTS_FLAG;
use crate::SECRETS_DIR_FLAG;
use account_utils::eth2_keystore::{keypair_from_secret, Keystore, KeystoreBuilder};
use account_utils::random_password;
use account_utils::{random_password, read_mnemonic_from_cli};
use clap::{App, Arg, ArgMatches};
use directory::ensure_dir_exists;
use directory::{parse_path_or_default_with_flag, DEFAULT_SECRET_DIR};
Expand Down
2 changes: 1 addition & 1 deletion account_manager/src/wallet/recover.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use crate::common::read_mnemonic_from_cli;
use crate::wallet::create::{create_wallet_from_mnemonic, STDIN_INPUTS_FLAG};
use crate::wallet::create::{HD_TYPE, NAME_FLAG, PASSWORD_FLAG, TYPE_FLAG};
use account_utils::read_mnemonic_from_cli;
use clap::{App, Arg, ArgMatches};
use std::path::PathBuf;

Expand Down
5 changes: 4 additions & 1 deletion book/src/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@
* [Run a Node](./run_a_node.md)
* [Become a Validator](./mainnet-validator.md)
* [Validator Management](./validator-management.md)
* [The `validator-manager` Command](./validator-manager.md)
* [Creating validators](./validator-manager-create.md)
* [Moving validators](./validator-manager-move.md)
* [Slashing Protection](./slashing-protection.md)
* [Voluntary Exits](./voluntary-exit.md)
* [Partial Withdrawals](./partial-withdrawal.md)
Expand Down Expand Up @@ -41,7 +44,7 @@
* [Remote Signing with Web3Signer](./validator-web3signer.md)
* [Database Configuration](./advanced_database.md)
* [Database Migrations](./database-migrations.md)
* [Key Management](./key-management.md)
* [Key Management (Deprecated)](./key-management.md)
* [Key Recovery](./key-recovery.md)
* [Advanced Networking](./advanced_networking.md)
* [Running a Slasher](./slasher.md)
Expand Down
27 changes: 24 additions & 3 deletions book/src/key-management.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,30 @@
# Key Management
# Key Management (Deprecated)

[launchpad]: https://launchpad.ethereum.org/

>
> **Note: While Lighthouse is able to generate the validator keys and the deposit data file to submit to the deposit contract, we strongly recommend using the [staking-deposit-cli](https://github.com/ethereum/staking-deposit-cli) to create validators keys and the deposit data file. This is because the [staking-deposit-cli](https://github.com/ethereum/staking-deposit-cli) has the option to assign a withdrawal address during the key generation process, while Lighthouse wallet will always generate keys with withdrawal credentials of type 0x00. This means that users who created keys using Lighthouse will have to update their withdrawal credentials in the future to enable withdrawals. In addition, Lighthouse generates the deposit data file in the form of `*.rlp`, which cannot be uploaded to the [Staking launchpad][launchpad] that accepts only `*.json` file. This means that users have to directly interact with the deposit contract to be able to submit the deposit if they were to generate the files using Lighthouse.**
**⚠️ The information on this page refers to tooling and process that have been deprecated. Please read the "Deprecation Notice". ⚠️**

## Deprecation Notice

This page recommends the use of the `lighthouse account-manager` tool to create
validators. This tool will always generate keys with the withdrawal credentials
of type `0x00`. This means the users who created keys using `lighthouse
account-manager` will have to update their withdrawal credentials in a
separate step to receive staking rewards.

In addition, Lighthouse generates the deposit data file in the form of `*.rlp`,
which cannot be uploaded to the [Staking launchpad][launchpad] that accepts only
`*.json` file. This means that users have to directly interact with the deposit
contract to be able to submit the deposit if they were to generate the files
using Lighthouse.

Rather than continuing to read this page, we recommend users visit either:

- The [Staking Launchpad][launchpad] for detailed, beginner-friendly instructions.
- The [staking-deposit-cli](https://github.com/ethereum/staking-deposit-cli) for a CLI tool used by the [Staking Launchpad][launchpad].
- The [validator-manager documentation](./validator-manager.md) for a Lighthouse-specific tool for streamlined validator management tools.

## The `lighthouse account-manager`

Lighthouse uses a _hierarchical_ key management system for producing validator
keys. It is hierarchical because each validator key can be _derived_ from a
Expand Down
4 changes: 4 additions & 0 deletions book/src/validator-management.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@ standard directories and do not start their `lighthouse vc` with the
this document. However, users with more complex needs may find this document
useful.

The [lighthouse validator-manager](./validator-manager.md) command can be used
to create and import validators to a Lighthouse VC. It can also be used to move
validators between two Lighthouse VCs.

## Introducing the `validator_definitions.yml` file

The `validator_definitions.yml` file is located in the `validator-dir`, which
Expand Down
Loading

0 comments on commit 1373dcf

Please sign in to comment.