Validator manager commands for the Keymanager APIs

book/src/SUMMARY.md

@@ -15,6 +15,7 @@
   * [The `validator-manager` Command](./validator-manager.md)
     * [Creating validators](./validator-manager-create.md)
     * [Moving validators](./validator-manager-move.md)
+    * [Managing validators](./validator-manager-api.md)
   * [Slashing Protection](./slashing-protection.md)
   * [Voluntary Exits](./voluntary-exit.md)
   * [Partial Withdrawals](./partial-withdrawal.md)

book/src/help_vm.md

@@ -23,6 +23,11 @@ Commands:
           "create-validators" command. This command only supports validators
           signing via a keystore on the local file system (i.e., not Web3Signer
           validators).
+  list
+          Lists all validators in a validator client using the HTTP API.
+  delete
+          Deletes one or more validators from a validator client using the HTTP
+          API.
   help
           Print this message or the help of the given subcommand(s)
 

book/src/help_vm_import.md

@@ -8,6 +8,11 @@ command.
 Usage: lighthouse validator_manager import [OPTIONS] --validators-file <PATH_TO_JSON_FILE>
 
 Options:
+      --builder-boost-factor <UINT64>
+          When provided, the imported validator will use this percentage
+          multiplier to apply to the builder's payload value when choosing
+          between a builder payload header and payload from the local execution
+          node.
   -d, --datadir <DIR>
           Used to specify a custom root data directory for lighthouse keys and
           databases. Defaults to $HOME/.lighthouse/{network} where network is
@@ -17,6 +22,10 @@ Options:
           Specifies the verbosity level used when emitting logs to the terminal.
           [default: info] [possible values: info, debug, trace, warn, error,
           crit]
+      --gas-limit <UINT64>
+          When provided, the imported validator will use this gas limit. It is
+          recommended to leave this as the default value by not specifying this
+          flag.
       --genesis-state-url <URL>
           A URL of a beacon-API compatible server from which to download the
           genesis state. Checkpoint sync server URLs can generally be used with
@@ -50,6 +59,11 @@ Options:
       --network <network>
           Name of the Eth2 chain Lighthouse will sync and follow. [possible
           values: mainnet, gnosis, chiado, sepolia, holesky]
+      --password <STRING>
+          Password of the keystore file.
+      --prefer-builder-proposals
+          When provided, the imported validator will always prefer blocks
+          constructed by builders, regardless of payload value.
       --safe-slots-to-import-optimistically <INTEGER>
           Used to coordinate manual overrides of the
           SAFE_SLOTS_TO_IMPORT_OPTIMISTICALLY parameter. This flag should only
@@ -58,6 +72,9 @@ Options:
           attack at the PoS transition block. Incorrect use of this flag can
           cause your node to possibly accept an invalid chain or sync more
           slowly. Be extremely careful with this flag.
+      --suggested-fee-recipient <ETH1_ADDRESS>
+          When provided, the imported validator will use the suggested fee
+          recipient. Omit this flag to use the default value from the VC.
   -t, --testnet-dir <DIR>
           Path to directory containing eth2_testnet specs. Defaults to a
           hard-coded Lighthouse testnet. Only effective if there is no existing
@@ -95,6 +112,9 @@ Options:
           http://localhost:5062]
 
 Flags:
+      --builder-proposals
+          When provided, the imported validator will attempt to create blocks
+          via builder rather than the local EL.
       --disable-log-timestamp
           If present, do not include timestamps in logging output.
       --disable-malloc-tuning
@@ -121,6 +141,9 @@ Flags:
           contain sensitive information about your validator and so this flag
           should be used with caution. For Windows users, the log file
           permissions will be inherited from the parent folder.
+      --standard-format
+          Use this flag when the validator keystore files are generated using
+          staking-deposit-cli or ethstaker-deposit-cli.
       --stdin-inputs
           If present, read all user inputs from stdin instead of tty.
 ```

book/src/validator-manager-api.md

@@ -0,0 +1,39 @@
+# Managing Validators
+
+The `lighthouse validator-manager` supports the [Keymanager API](https://ethereum.github.io/keymanager-APIs/#/) to list, import and delete keystores via the HTTP API. This requires the validator client running with the flag `--http`.
+
+## Delete
+
+The `delete` command deletes one or more validators from the validator client. It will also modify the `validator_definitions.yml` file automatically so there is no manual action required from the user after the delete. To `delete`:
+
+```bash
+lighthouse vm delete --vc-token <API-TOKEN-PATH> --validators pubkey1, pubkey2
+```
+
+Example:
+
+```bash
+lighthouse vm remove --vc-token ~/.lighthouse/mainnet/validators/api-token.txt --validators 0x8885c29b8f88ee9b9a37b480fd4384fed74bda33d85bc8171a904847e65688b6c9bb4362d6597fd30109fb2def6c3ae4, 0xa262dae3dcd2b2e280af534effa16bedb27c06f2959e114d53bd2a248ca324a018dc73179899a066149471a94a1bc92f
+```
+
+## Import
+
+The `import` command imports validator keystores generated by the staking-deposit-cli. To import validator keystore:
+
+```bash
+lighthouse vm import --standard-format --vc-token <API-TOKEN-PATH> --validators-file /path/to/json --password keystore_password
+```
+
+Example:
+
+```
+lighthouse vm import --standard-format --vc-token ~/.lighthouse/mainnet/validators/api-token.txt --validator-file keystore.json --password keystore_password
+```
+
+## List
+
+To list the validators running on the validator client:
+
+```bash
+lighthouse vm list --vc-token ~/.lighthouse/mainnet/validators/api-token.txt
+```

book/src/validator-manager-create.md

@@ -69,6 +69,8 @@ lighthouse \
 > Be sure to remove `./validators.json` after the import is successful since it
 > contains unencrypted validator keystores.
 
+> Note: To import validators with validator-manager using keystore files created using the staking deposit cli, refer to [Managing Validators](./validator-manager-api.md#import).
+
 ## Detailed Guide
 
 This guide will create two validators and import them to a VC. For simplicity,

common/account_utils/src/lib.rs

@@ -9,13 +9,16 @@ use eth2_wallet::{
 use filesystem::{create_with_600_perms, Error as FsError};
 use rand::{distributions::Alphanumeric, Rng};
 use serde::{Deserialize, Serialize};
-use std::fs::{self, File};
 use std::io;
 use std::io::prelude::*;
 use std::path::{Path, PathBuf};
 use std::str::from_utf8;
 use std::thread::sleep;
 use std::time::Duration;
+use std::{
+    fs::{self, File},
+    str::FromStr,
+};
 use zeroize::Zeroize;
 
 pub mod validator_definitions;
@@ -215,6 +218,14 @@ pub fn mnemonic_from_phrase(phrase: &str) -> Result<Mnemonic, String> {
 #[serde(transparent)]
 pub struct ZeroizeString(String);
 
+impl FromStr for ZeroizeString {
+    type Err = String;
+
+    fn from_str(s: &str) -> Result<Self, Self::Err> {
+        Ok(Self(s.to_owned()))
+    }
+}
+
 impl From<String> for ZeroizeString {
     fn from(s: String) -> Self {
         Self(s)

lighthouse/tests/validator_manager.rs

@@ -203,6 +203,14 @@ pub fn validator_import_defaults() {
                 vc_url: SensitiveUrl::parse("http://localhost:5062").unwrap(),
                 vc_token_path: PathBuf::from("./token.json"),
                 ignore_duplicates: false,
+                password: None,
+                fee_recipient: None,
+                builder_boost_factor: None,
+                gas_limit: None,
+                builder_proposals: Some(false),
+                enabled: None,
+                prefer_builder_proposals: Some(false),
+                standard_format: false,
             };
             assert_eq!(expected, config);
         });
@@ -220,6 +228,14 @@ pub fn validator_import_misc_flags() {
                 vc_url: SensitiveUrl::parse("http://localhost:5062").unwrap(),
                 vc_token_path: PathBuf::from("./token.json"),
                 ignore_duplicates: true,
+                password: None,
+                fee_recipient: None,
+                builder_boost_factor: None,
+                gas_limit: None,
+                builder_proposals: Some(false),
+                enabled: None,
+                prefer_builder_proposals: Some(false),
+                standard_format: false,
             };
             assert_eq!(expected, config);
         });

validator_client/src/http_api/keystores.rs

@@ -75,12 +75,6 @@ pub fn import<T: SlotClock + 'static, E: EthSpec>(
         )));
     }
 
-    info!(
-        log,
-        "Importing keystores via standard HTTP API";
-        "count" => request.keystores.len(),
-    );
-
     // Import slashing protection data before keystores, so that new keystores don't start signing
     // without it. Do not return early on failure, propagate the failure to each key.
     let slashing_protection_status =
@@ -156,6 +150,19 @@ pub fn import<T: SlotClock + 'static, E: EthSpec>(
         statuses.push(status);
     }
 
+    let successful_import = statuses
+        .iter()
+        .filter(|status| matches!(status.status, ImportKeystoreStatus::Imported))
+        .count();
+
+    if successful_import > 0 {
+        info!(
+            log,
+            "Imported keystores via standard HTTP API";
+            "count" => successful_import,
+        );
+    }
+
     Ok(ImportKeystoresResponse { data: statuses })
 }
 
@@ -238,7 +245,23 @@ pub fn delete<T: SlotClock + 'static, E: EthSpec>(
     task_executor: TaskExecutor,
     log: Logger,
 ) -> Result<DeleteKeystoresResponse, Rejection> {
-    let export_response = export(request, validator_store, task_executor, log)?;
+    let export_response = export(request, validator_store, task_executor, log.clone())?;
+
+    // Check the status is Deleted to confirm deletion is successful, then only display the log
+    let successful_deletion = export_response
+        .data
+        .iter()
+        .filter(|response| matches!(response.status.status, DeleteKeystoreStatus::Deleted))
+        .count();
+
+    if successful_deletion > 0 {
+        info!(
+            log,
+            "Deleted keystore via standard HTTP API";
+            "count" => successful_deletion,
+        );
+    }
+
     Ok(DeleteKeystoresResponse {
         data: export_response
             .data

validator_manager/Cargo.toml

@@ -20,6 +20,7 @@ tree_hash = { workspace = true }
 eth2 = { workspace = true }
 hex = { workspace = true }
 tokio = { workspace = true }
+derivative = { workspace = true }
 
 [dev-dependencies]
 tempfile = { workspace = true }

validator_manager/src/create_validators.rs

@@ -45,15 +45,6 @@ pub fn cli_app() -> Command {
                 Another, optional JSON file is created which contains a list of validator \
                 deposits in the same format as the \"ethereum/staking-deposit-cli\" tool.",
         )
-        .arg(
-            Arg::new("help")
-                .long("help")
-                .short('h')
-                .help("Prints help information")
-                .action(ArgAction::HelpLong)
-                .display_order(0)
-                .help_heading(FLAG_HEADER),
-        )
         .arg(
             Arg::new(OUTPUT_PATH_FLAG)
                 .long(OUTPUT_PATH_FLAG)