Scanner Integration Options

[upbqdn]

Status Quo

The Zebra monorepo currently contains two new crates:

• zebra-scan, and
• zebra-grpc

that get compiled into the zebrad binary when the shielded-scan feature is enabled. The crate zebra-grpc is a dependency of zebra-scan, and no other crate currently depends on it.

Options for Decoupling The scanner from zebrad

Some arguments for decoupling:

1. Timing attacks on private viewing keys: Run the scanner in an isolated environment #8141, security: Run the scanner it is own tokio executor, and use low-priority threads to avoid timing attacks #8043.
2. Memory safety (particularly C++ code), and private key file access via remote code execution.
3. Process isolation.
4. Maintenance: tightly binding the scanner to Zebra’s internal APIs.

The scanner can stay in zebrad, or can be decoupled from it in the following ways.

Standalone Binary

The scanner could be a separate binary from zebrad that would directly open zebrad's finalized state in a read-only mode. This will work because the finalized state is an append-only (implying immutability of existing data) RocksDB database, and RocksDB supports multiple read-only concurrent instances. This way, there wouldn't need to be any communication protocol between zebrad and the scanner as far as the finalized state is concerned. The scanner would indeed reuse Zebra's crates for the types it needs.

The non-finalized state is ~100 blocks ahead of the finalized state and lives only in zebrad's memory. To scan it, the scanner would simply fetch the blocks via the existing JSON RPCs.

Scanner Based on lightwalletd Instead of zebrad

The scanner could also be either

1. implemented in lightwalletd, or
2. connected to lightwalletd as a standalone binary.

This should work because the current scanner converts all transactions to the compact format used by lightwalletd before scanning them, and it looks like it won't need any extra data that lightwalletd couldn't provide. If the scanner is directly implemented in lightwalletd, there wouldn't need to be a standalone gRPC frontend since lightwalletd already has one. A single gRPC frontend would be easier for users since they would need to obtain a single TLS certificate instead of two.

Note that lightwalletd is implemented in Go, so yet another option is to re-implement lightwalletd in Rust and include the scanner there. Such a lightwalletd implementation might not need its own separate persistent storage, but it might utilize zebrad's state and perform compact-block conversion on the fly, similarly to what the scanner currently does. There also wouldn't need to be the initial sync, and lightwalletd would be a wrapper around zebrads state providing a gRPC endpoint.

Options for the gRPC endpoint

There are also multiple places where the gRPC endpoint can be placed:

1. The zebrad binary as it is now.
2. If the scanner ever runs in its own binary, it would be suitable to place the gRPC server there.
3. If the scanner is ever implemented in lightwalletd, there wouldn't need to be a new gRPC server.
4. It's own binary communicating with the scanner and possibly zebrad and lightwalletd, unifying remote access to all those services.