Additional changes to delegate bitcoin core keys to eclair (#2726)

Refactor the `BitcoinCoreClient` and `LocalOnChainKeyManager` to:

- rely less on exceptions
- use more idiomatic scala (reduce dependency on kotlin types)
- provide more detailed logs

We also simplify the `useEclairSigner` field in `BitcoinCoreClient`.
The complexity of handling the case where there was an on-chain key
manager but for a different wallet than the one configured isn't
something that should be used, so it wasn't worth supporting.

Some checks were inconsistent and are now unified:

- checking the exact `scriptPubKey` in our outputs in TODO and TODO
- we allow using `fundTransaction` with a tx that already includes a
  change output (which may happen when RBF-ing a transaction)
- `getP2wpkhPubkeyHashForChange` didn't verify the returned key

We completely separate the two cases in `signPsbt`, because otherwise
in the non eclair-backed case, we were calling bitcoind's `processpsbt`
twice for no good reason, which is bad for performance.

We also decouple the `OnChainKeyManager` from the `BitcoinCoreClient`.
This lets users keep running their eclair node with a bitcoin client that
owns the private key while configuring the on-chain key manager for a
future bitcoin client that will leverage this on-chain key manager.

Users can use the eclair APIs to get the master xpub and descriptors to
properly configure their next bitcoin core node, and switch to it once it
has synchronized the descriptors.

docs/BitcoinCoreKeys.md

@@ -6,22 +6,22 @@ is less secure than for Eclair (because it is shared among several services for
 
 ## Configuring Eclair and Bitcoin Core to use a new Eclair-backed bitcoin wallet
 
-Follow these steps to delegate onchain key management to eclair:
+Follow these steps to delegate on-chain key management to eclair:
 
-1) Generate a BIP39 mnemonic code and passphrase
+### 1. Generate a BIP39 mnemonic code and passphrase
 
 You can use any BIP39-compatible tool, including most hardware wallets.
 
-2) Create an `eclair-signer.conf` configuration file add it to eclair's data directory
+### 2. Create an `eclair-signer.conf` configuration file add it to eclair's data directory
 
 A signer configuration file uses the HOCON format that we already use for `eclair.conf` and must include the following options:
 
- key                      | description                                                              
+ key                      | description
 --------------------------|--------------------------------------------------------------------------
- eclair.signer.wallet     | wallet name                                                              
- eclair.signer.mnemonics  | BIP39 mnemonic words                                                     
- eclair.signer.passphrase | passphrase                                                               
- eclair.signer.timestamp  | wallet creation UNIX timestamp. Set to the current time for new wallets. 
+ eclair.signer.wallet     | wallet name
+ eclair.signer.mnemonics  | BIP39 mnemonic words
+ eclair.signer.passphrase | passphrase
+ eclair.signer.timestamp  | wallet creation UNIX timestamp. Set to the current time for new wallets.
 
 This is an example of `eclair-signer.conf` configuration file:
 
@@ -38,7 +38,7 @@ This is an example of `eclair-signer.conf` configuration file:
 }
 ```
 
-3) Configure Eclair to handle private keys for this wallet
+### 3. Configure Eclair to handle private keys for this wallet
 
 Set `eclair.bitcoind.wallet` to the name of the wallet in your `eclair-signer.conf` file (`eclair` in the example above) and restart Eclair.
 Eclair will automatically create a new, empty, descriptor-enabled, watch-only wallet in Bitcoin Core and import its descriptors.
@@ -48,45 +48,38 @@ passphrase that your are using are new, you can safely update this timestamp, bu
 the steps described in the next section.
 
 You now have a Bitcoin Core watch-only wallet for which only your Eclair node can sign transactions. This Bitcoin Core wallet can
-safely be copied to another Bitcoin Core node to monitor your onchain funds.
+safely be copied to another Bitcoin Core node to monitor your on-chain funds.
 
 You can also use `eclair-cli getmasterxpub` to get a BIP32 extended public key that you can import into any compatible Bitcoin wallet
 to create a watch-only wallet (Electrum for example) that you can use to monitor your Bitcoin Core balance.
 
 :warning: this means that your Bitcoin Core wallet cannot send funds on its own (since it cannot access private keys to sign transactions).
-To send funds onchain you must use `eclair-cli sendonchain`.
+To send funds on-chain you must use `eclair-cli sendonchain`.
 
 :warning: to backup the private keys of this wallet you must either backup your mnemonic code and passphrase, or backup the `eclair-signer.conf` file in your eclair
 directory (default is `~/.eclair`) along with your channels and node seed files.
 
-:warning: You can also initialize a backup onchain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up
+:warning: You can also initialize a backup on-chain wallet with the same mnemonic code and passphrase (on a hardware wallet for example), but be warned that using them may interfere with your node's operations (for example you may end up
 double-spending funding transactions generated by your node).
 
-## Importing an existing Eclair-backed bitcoin core wallet
+## Importing an existing Eclair-backed Bitcoin Core wallet
 
-The steps above describe how you can simply switch to a new Eclair-backed bitcoin wallet.
-If you are already using an Eclair-backed bitcoin wallet that you want to move to another setup, you have several options:
+The steps above described how you can simply switch to a new Eclair-backed bitcoin wallet.
+Follow the steps below if you are already using an Eclair-backed bitcoin wallet that you want to move to another Bitcoin Core node.
 
-### Copy the bitcoin core wallet and `eclair-signer.conf`
+### 1. Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core
 
-Copy your wallet file to a new Bitcoin Core node, and load it with `bitcoin-cli loadwallet`. Bitcoin Core may need to scan the blockchain which could take some time.
-Copy `eclair-signer.conf` to your Eclair data directory and set `eclair.bitcoind.wallet` to the name of the wallet configured in `eclair-signer.conf` (and which should also be the name of your Bitcoin Core wallet).
-Once your wallet has been imported, just restart Eclair.
+Start by creating a watch-only wallet on your new Bitcoin Core node.
 
-### Create an empty wallet manually and import Eclair descriptors
-
-1) Create an empty, descriptor-enabled, watch-only wallet in Bitcoin Core:
-   :warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair")
+:warning: The name must match the one that you set in `eclair-signer.conf` (here we use "eclair")
 
 ```shell
 $ bitcoin-cli -named createwallet wallet_name=eclair disable_private_keys=true blank=true descriptors=true load_on_startup=true
 ```
 
-2) Import public descriptors generated by Eclair
-
-Copy `eclair-signer.conf` to your Eclair data directory but do not change `eclair.bitcoind.wallet`, and restart Eclair.
+### 2. Import public descriptors generated by Eclair
 
-`eclair-cli getdescriptors` will return public wallet descriptors in a format that is compatible with Bitcoin Core, and that you can import with `bitcoin-cli -rpcwallet=eclair importdescriptors`
+Calling `eclair-cli getdescriptors` on your existing Eclair node will return public wallet descriptors in a format that is compatible with Bitcoin Core.
 This is an example of descriptors generated by Eclair:
 
 ```json
@@ -106,7 +99,7 @@ This is an example of descriptors generated by Eclair:
 ]
 ```
 
-You can generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands:
+Generate the descriptors with your Eclair node and import them into a Bitcoin node with the following commands:
 
 ```shell
 $ eclair-cli getdescriptors | jq --raw-output -c > descriptors.json
@@ -115,6 +108,15 @@ $ cat descriptors.json | xargs -0 bitcoin-cli -rpcwallet=eclair importdescriptor
 
 :warning: Importing descriptors can take a long time, and your Bitcoin Core node will not be usable until it's done
 
-3) Configure Eclair to handle private keys for this wallet
+### 3. Configure Eclair to use your new Bitcoin Core node
+
+Once your new Bitcoin Core node has finished importing the descriptors, it is ready to be used by Eclair.
+
+In your `eclair.conf`:
+
+- set `eclair.bitcoind.host` to the address of your new Bitcoin Core node
+- set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf`
+- set `eclair.bitcoind.zmqblock` and `eclair.bitcoind.zmqtx` to use your new Bitcoin Core node
+- update other field in the `eclair.bitcoind` section if necessary (`rpcport`, `auth`, `rpcuser`, `rpcuser`, etc)
 
-Set `eclair.bitcoind.wallet` to the name of the wallet in `eclair-signer.conf` and restart Eclair.
+Restart Eclair and it will start using your new Bitcoin Core node.

eclair-core/src/main/scala/fr/acinq/eclair/Eclair.scala

@@ -178,7 +178,7 @@ trait Eclair {
 
   def payOfferBlocking(offer: Offer, amount: MilliSatoshi, quantity: Long, externalId_opt: Option[String] = None, maxAttempts_opt: Option[Int] = None, maxFeeFlat_opt: Option[Satoshi] = None, maxFeePct_opt: Option[Double] = None, pathFindingExperimentName_opt: Option[String] = None, connectDirectly: Boolean = false)(implicit timeout: Timeout): Future[PaymentEvent]
 
-  def getOnchainMasterPubKey(account: Long): String
+  def getOnChainMasterPubKey(account: Long): String
 
   def getDescriptors(account: Long): Descriptors
 
@@ -669,16 +669,16 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging {
     }
   }
 
-  def payOfferInternal(offer: Offer,
-                       amount: MilliSatoshi,
-                       quantity: Long,
-                       externalId_opt: Option[String],
-                       maxAttempts_opt: Option[Int],
-                       maxFeeFlat_opt: Option[Satoshi],
-                       maxFeePct_opt: Option[Double],
-                       pathFindingExperimentName_opt: Option[String],
-                       connectDirectly: Boolean,
-                       blocking: Boolean)(implicit timeout: Timeout): Future[Any] = {
+  private def payOfferInternal(offer: Offer,
+                               amount: MilliSatoshi,
+                               quantity: Long,
+                               externalId_opt: Option[String],
+                               maxAttempts_opt: Option[Int],
+                               maxFeeFlat_opt: Option[Satoshi],
+                               maxFeePct_opt: Option[Double],
+                               pathFindingExperimentName_opt: Option[String],
+                               connectDirectly: Boolean,
+                               blocking: Boolean)(implicit timeout: Timeout): Future[Any] = {
     if (externalId_opt.exists(_.length > externalIdMaxLength)) {
       return Future.failed(new IllegalArgumentException(s"externalId is too long: cannot exceed $externalIdMaxLength characters"))
     }
@@ -721,14 +721,14 @@ class EclairImpl(appKit: Kit) extends Eclair with Logging {
     payOfferInternal(offer, amount, quantity, externalId_opt, maxAttempts_opt, maxFeeFlat_opt, maxFeePct_opt, pathFindingExperimentName_opt, connectDirectly, blocking = true).mapTo[PaymentEvent]
   }
 
-  override def getDescriptors(account: Long): Descriptors = appKit.wallet match {
-    case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getDescriptors(account)
-    case _ => throw new RuntimeException("onchain seed is not configured")
+  override def getDescriptors(account: Long): Descriptors = appKit.nodeParams.onChainKeyManager_opt match {
+    case Some(keyManager) => keyManager.descriptors(account)
+    case _ => throw new RuntimeException("on-chain seed is not configured")
   }
 
-  override def getOnchainMasterPubKey(account: Long): String = appKit.wallet match {
-    case bitcoinCoreClient: BitcoinCoreClient if bitcoinCoreClient.onchainKeyManager_opt.isDefined => bitcoinCoreClient.onchainKeyManager_opt.get.getOnchainMasterPubKey(account)
-    case _ => throw new RuntimeException("onchain seed is not configured")
+  override def getOnChainMasterPubKey(account: Long): String = appKit.nodeParams.onChainKeyManager_opt match {
+    case Some(keyManager) => keyManager.masterPubKey(account)
+    case _ => throw new RuntimeException("on-chain seed is not configured")
   }
 
   override def stop(): Future[Unit] = {

eclair-core/src/main/scala/fr/acinq/eclair/NodeParams.scala

@@ -25,16 +25,16 @@ import fr.acinq.eclair.channel.ChannelFlags
 import fr.acinq.eclair.channel.fsm.Channel
 import fr.acinq.eclair.channel.fsm.Channel.{ChannelConf, UnhandledExceptionStrategy}
 import fr.acinq.eclair.crypto.Noise.KeyPair
-import fr.acinq.eclair.crypto.keymanager.{ChannelKeyManager, NodeKeyManager}
+import fr.acinq.eclair.crypto.keymanager.{ChannelKeyManager, NodeKeyManager, OnChainKeyManager}
 import fr.acinq.eclair.db._
 import fr.acinq.eclair.io.MessageRelay.{RelayAll, RelayChannelsOnly, RelayPolicy}
 import fr.acinq.eclair.io.PeerConnection
 import fr.acinq.eclair.message.OnionMessages.OnionMessageConfig
 import fr.acinq.eclair.payment.relay.Relayer.{AsyncPaymentsParams, RelayFees, RelayParams}
 import fr.acinq.eclair.router.Announcements.AddressException
 import fr.acinq.eclair.router.Graph.{HeuristicsConstants, WeightRatios}
+import fr.acinq.eclair.router.Router._
 import fr.acinq.eclair.router.{Graph, PathFindingExperimentConf}
-import fr.acinq.eclair.router.Router.{MessageRouteParams, MultiPartParams, PathFindingConf, RouterConf, SearchBoundaries}
 import fr.acinq.eclair.tor.Socks5ProxyParams
 import fr.acinq.eclair.wire.protocol._
 import grizzled.slf4j.Logging
@@ -54,6 +54,7 @@ import scala.jdk.CollectionConverters._
  */
 case class NodeParams(nodeKeyManager: NodeKeyManager,
                       channelKeyManager: ChannelKeyManager,
+                      onChainKeyManager_opt: Option[OnChainKeyManager],
                       instanceId: UUID, // a unique instance ID regenerated after each restart
                       private val blockHeight: AtomicLong,
                       private val feerates: AtomicReference[FeeratesPerKw],
@@ -101,7 +102,7 @@ case class NodeParams(nodeKeyManager: NodeKeyManager,
   def currentFeerates: FeeratesPerKw = feerates.get()
 
   /** Only to be used in tests. */
-  def setFeerates(value: FeeratesPerKw) = feerates.set(value)
+  def setFeerates(value: FeeratesPerKw): Unit = feerates.set(value)
 
   /** Returns the features that should be used in our init message with the given peer. */
   def initFeaturesFor(nodeId: PublicKey): Features[InitFeature] = overrideInitFeatures.getOrElse(nodeId, features).initFeatures()
@@ -211,7 +212,8 @@ object NodeParams extends Logging {
     }
   }
 
-  def makeNodeParams(config: Config, instanceId: UUID, nodeKeyManager: NodeKeyManager, channelKeyManager: ChannelKeyManager,
+  def makeNodeParams(config: Config, instanceId: UUID,
+                     nodeKeyManager: NodeKeyManager, channelKeyManager: ChannelKeyManager, onChainKeyManager_opt: Option[OnChainKeyManager],
                      torAddress_opt: Option[NodeAddress], database: Databases, blockHeight: AtomicLong, feerates: AtomicReference[FeeratesPerKw],
                      pluginParams: Seq[PluginParams] = Nil): NodeParams = {
     // check configuration for keys that have been renamed
@@ -475,6 +477,7 @@ object NodeParams extends Logging {
     NodeParams(
       nodeKeyManager = nodeKeyManager,
       channelKeyManager = channelKeyManager,
+      onChainKeyManager_opt = onChainKeyManager_opt,
       instanceId = instanceId,
       blockHeight = blockHeight,
       feerates = feerates,