Unable to build reth images (#27)

* 🌱 Add snapshot support * 🔨 Add retry mechanism for snapshot download & integrity verification * 🐳 Remove environment from docker-compose.yml * ⬆️ Bump op-reth to v1.0.3 * 🐳 Add env flag support to choose reth build profile
LiskHQ · Jul 31, 2024 · 1d212a8 · 1d212a8
1 parent 77a8762
commit 1d212a8
Show file tree

Hide file tree

Showing 8 changed files with 111 additions and 127 deletions.
diff --git a/README.md b/README.md
@@ -24,16 +24,16 @@ The following system requirements are recommended to run a Lisk L2 node.
 
 ## Supported networks
 
-| Network              | Status |
-| -------------------- | ------ |
-| Lisk Sepolia Testnet | ✅     |
-| Lisk Mainnet         | ✅     |
+| Network      | Status |
+| ------------ | ------ |
+| Lisk Sepolia | ✅     |
+| Lisk Mainnet | ✅     |
 
 ## Usage
 
 > **Note**:
-> <br>It is currently not possible to run the nodes with the `--op-network` flag until the configs for Lisk have been merged to the [superchain-registry](https://github.com/ethereum-optimism/superchain-registry).
-> <br>Currently, due to ongoing changes in the above repo, addition of new chains to the registry have been paused. Once the maintenance is over, we will submit PRs to add the config for the Lisk Mainnet and Lisk Sepolia Testnet.
+> - It is currently not possible to run the nodes with the `--op-network` flag until the configs for Lisk have been merged to the [superchain-registry](https://github.com/ethereum-optimism/superchain-registry).
+> - Currently, due to ongoing changes in the above repo, addition of new chains to the registry have been paused. Once the maintenance is over, we will submit PRs to add the config for Lisk Mainnet and Lisk Sepolia.
 
 ### Clone the Repository
 
@@ -46,16 +46,25 @@ cd lisk-node
 
 1. Ensure you have an Ethereum L1 full node RPC available (not Lisk), and set the `OP_NODE_L1_ETH_RPC` and the `OP_NODE_L1_BEACON` variables (within the `.env.*` files, if using docker-compose). If running your own L1 node, it needs to be synced before the Lisk node will be able to fully sync.
 
-2. Please ensure that the environment file relevant to your network (`.env.sepolia`, or `.env.mainnet`) is set for the `env_file` properties within `docker-compose.yml`. By default, it is set to `.env.mainnet`.
+1. Please ensure that the environment file relevant to your network (`.env.sepolia`, or `.env.mainnet`) is set for the `env_file` properties within `docker-compose.yml`. By default, it is set to `.env.mainnet`.
 
-3. We currently support running either the `op-geth` or the `op-reth` nodes alongside the `op-node`. By default, we run the `op-geth` node. If you would like to run the `op-reth` node instead, please set the `CLIENT` environment variable to `reth` before starting the node.
+1. We currently support running either the `op-geth` or the `op-reth` nodes alongside the `op-node`. By default, we run the `op-geth` node. If you would like to run the `op-reth` client, please set the `CLIENT` environment variable to `reth` before starting the node.
+    > **Note**:
+    > - The `op-reth` client can be built in either the `release` (default) or `maxperf` profile. To learn more about them, please check reth's documentation on [Optimizations](https://github.com/paradigmxyz/reth/blob/main/book/installation/source.md#optimizations). Please set the `RETH_BUILD_PROFILE` environment variable accordingly.
+    > - If you are building the `op-reth` client in `maxperf` profile, please ensure that you have a machine with 32 GB RAM.
+    > - Additionally, if you have the Docker Desktop installed on your system, please make sure to set `Memory limit` to a minimum of `16 GB`.<br>It can be set under `Settings -> Resources -> Resource Allocation -> Memory limit`.
 
-4. Run:
+1. Run:
+    <br>with `op-geth`:
     ```sh
     docker compose up --build --detach
     ```
+    or, with `op-reth`:
+    ```sh
+    CLIENT=reth RETH_BUILD_PROFILE=release docker compose up --build --detach
+    ```
 
-5. You should now be able to `curl` your Lisk node:
+1. You should now be able to `curl` your Lisk node:
     ```sh
     curl -s -d '{"id":0,"jsonrpc":"2.0","method":"eth_getBlockByNumber","params":["latest",false]}' \
       -H "Content-Type: application/json" http://localhost:8545
@@ -105,49 +114,13 @@ Navigate to your `op-geth` directory and initialize the service by running the c
 ```
 
 > **Note**:
-> <br>Alternatively, this initialization step can be skipped by specifying `--op-network=OP_NODE_NETWORK` flag in the start commands below.
-> <br>This flag automatically fetches the necessary information from the [superchain-registry](https://github.com/ethereum-optimism/superchain-registry).
+> - Alternatively, this initialization step can be skipped by specifying `--op-network=OP_NODE_NETWORK` flag in the start commands below.
+> - This flag automatically fetches the necessary information from the [superchain-registry](https://github.com/ethereum-optimism/superchain-registry).
 
 #### Run op-geth
 
 Navigate to your `op-geth` directory and start service by running the command:
 
-For, Lisk Sepolia Testnet:
-
-```sh
-./build/bin/geth \
-    --datadir=$DATADIR_PATH \
-    --verbosity=3 \
-    --http \
-    --http.corsdomain="*" \
-    --http.vhosts="*" \
-    --http.addr=0.0.0.0 \
-    --http.port=8545 \
-    --http.api=web3,debug,eth,net,engine \
-    --authrpc.addr=0.0.0.0 \
-    --authrpc.port=8551 \
-    --authrpc.vhosts="*" \
-    --authrpc.jwtsecret=PATH_TO_JWT_TEXT_FILE \
-    --ws \
-    --ws.addr=0.0.0.0 \
-    --ws.port=8546 \
-    --ws.origins="*" \
-    --ws.api=debug,eth,net,engine \
-    --metrics \
-    --metrics.addr=0.0.0.0 \
-    --metrics.port=6060 \
-    --syncmode=full \
-    --gcmode=full \
-    --maxpeers=100 \
-    --nat=extip:0.0.0.0 \
-    --rollup.sequencerhttp=SEQUENCER_HTTP \
-    --rollup.halt=major \
-    --port=30303 \
-    --rollup.disabletxpoolgossip=true
-```
-
-For, Lisk Mainnet:
-
 ```sh
 ./build/bin/geth \
     --datadir=$DATADIR_PATH \
@@ -184,18 +157,17 @@ Refer to the `op-geth` configuration [documentation](https://docs.optimism.io/bu
 
 #### Run op-reth
 
-> **Note**:
-> <br>Currently we do not support running the reth client on the Lisk Sepolia Testnet.
-
 Navigate to your `reth` directory and start service by running the command:
 
-For, Lisk Mainnet:
-
 ```sh
 ./target/release/op-reth node \
   -vvv \
+  --chain=PATH_TO_NETWORK_GENESIS_FILE \
   --datadir="$DATADIR_PATH" \
   --log.stdout.format log-fmt \
+  --authrpc.addr=0.0.0.0 \
+  --authrpc.port=8551 \
+  --authrpc.jwtsecret=PATH_TO_JWT_TEXT_FILE \
   --ws \
   --ws.origins="*" \
   --ws.addr=0.0.0.0 \
@@ -205,41 +177,25 @@ For, Lisk Mainnet:
   --http.corsdomain="*" \
   --http.addr=0.0.0.0 \
   --http.port=8545 \
-  --http.api=debug,eth,net,txpool \
-  --authrpc.addr=0.0.0.0 \
-  --authrpc.port=8551 \
-  --authrpc.jwtsecret=PATH_TO_JWT_TEXT_FILE \
+  --http.api=web3,debug,eth,net,txpool \
   --metrics=0.0.0.0:6060 \
-  --chain=PATH_TO_NETWORK_GENESIS_FILE \
   --disable-discovery \
+  --port=30303 \
   --rollup.sequencer-http=SEQUENCER_HTTP \
   --rollup.disable-tx-pool-gossip
 ```
 
 Refer to the `reth` configuration [documentation](https://reth.rs/cli/reth/node.html#reth-node) for detailed information about available options.
 
-> **Note**: Official Lisk sequencer HTTP:
-> <br>For Mainnet: https://rpc.sepolia-api.lisk.com
-> <br>For Sepolia Testnet: https://rpc.api.lisk.com
+> **Note**:
+> <br>Official Lisk Sequencer HTTP RPC:
+> - **Lisk Mainnet**: https://rpc.sepolia-api.lisk.com
+> - **Lisk Sepolia**: https://rpc.api.lisk.com
 
 #### Run op-node
 
 Navigate to your `op-node` directory and start service by running the command:
 
-For, Lisk Sepolia Testnet:
-
-```sh
-./bin/op-node \
-  --l1=$OP_NODE_L1_ETH_RPC \
-  --l1.rpckind=$OP_NODE_L1_RPC_KIND \
-  --l1.beacon=$OP_NODE_L1_BEACON \
-  --l2=ws://localhost:8551 \
-  --l2.jwt-secret=PATH_TO_JWT_TEXT_FILE \
-  --rollup.config=PATH_TO_NETWORK_ROLLUP_FILE
-```
-
-For, Lisk Mainnet:
-
 ```sh
 ./bin/op-node \
   --l1=$OP_NODE_L1_ETH_RPC \
@@ -259,8 +215,8 @@ INFO [06-26|13:31:20.389] Advancing bq origin                      origin=17171d
 For more information refer to the OP [documentation](https://docs.optimism.io/builders/node-operators/tutorials/mainnet#full-sync).
 
 > **Note**:
-> <br>In case you had skipped the `op-geth` [initialization step](#initialize-op-geth) above, you can start the node with the `--network=OP_NODE_NETWORK` flag.
-> <br>When specifying the `--network` flag, kindly make sure to remove the `--rollup.config` flag.
+> - In case you had skipped the `op-geth` [initialization step](#initialize-op-geth) above, you can start the node with the `--network=OP_NODE_NETWORK` flag.
+> - When specifying the `--network` flag, kindly make sure to remove the `--rollup.config` flag.
 
 Refer to the `op-node` configuration [documentation](https://docs.optimism.io/builders/node-operators/management/configuration#op-node) for detailed information about available options.
 
@@ -306,4 +262,4 @@ echo Latest synced block behind by: \
 $((($( date +%s )-\
 $( curl -s -d '{"id":0,"jsonrpc":"2.0","method":"optimism_syncStatus"}' -H "Content-Type: application/json" http://localhost:9545 |
    jq -r .result.unsafe_l2.timestamp))/60)) minutes
-```
+```
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -4,6 +4,8 @@ services:
     build:
       context: .
       dockerfile: ${CLIENT}/Dockerfile
+      args:
+        PROFILE: ${RETH_BUILD_PROFILE:-release}
     ports:
       - "8545:8545"       # RPC
       - "8546:8546"       # websocket
@@ -13,13 +15,15 @@ services:
         - ${HOST_DATA_DIR}:/data
     env_file:
       # select your network here:
-      #   - .env.sepolia
+      # - .env.sepolia
       - .env.mainnet
   node:
     platform: linux/amd64
     build:
       context: .
       dockerfile: ${CLIENT}/Dockerfile
+      args:
+        PROFILE: ${RETH_BUILD_PROFILE:-release}
     depends_on:
       - execution
     ports:
@@ -31,5 +35,5 @@ services:
     command: [ "bash", "./op-node-entrypoint" ]
     env_file:
       # select your network here:
-      #   - .env.sepolia
+      # - .env.sepolia
       - .env.mainnet
diff --git a/geth/Dockerfile b/geth/Dockerfile
@@ -9,9 +9,9 @@ RUN git clone $REPO --branch op-node/$VERSION --single-branch . && \
     git switch -c branch-$VERSION && \
     bash -c '[ "$(git rev-parse HEAD)" = "$COMMIT" ]'
 
-# Temporary patch until our Raas provider redeploys the SystemConfig contract
-COPY geth/lisk-hotfix.patch .
-RUN git apply lisk-hotfix.patch && \
+# Temporary patch until our RaaS provider re-deploys the SystemConfig contract
+COPY op-node-lisk-hotfix.patch .
+RUN git apply op-node-lisk-hotfix.patch && \
     cd op-node && \
     make VERSION=$VERSION op-node
 

diff --git a/geth/geth-entrypoint b/geth/geth-entrypoint
@@ -25,6 +25,7 @@ OP_GETH_SYNCMODE="${OP_GETH_SYNCMODE:-full}"
 #     exit 1
 # fi
 
+# Ensure data directory exists
 mkdir -p $GETH_DATA_DIR
 
 # Init genesis
@@ -58,16 +59,10 @@ if [ "${HOST_IP:+x}" = x ]; then
 fi
 
 # Start service
-echoBanner "Starting node..."
+echoBanner "Starting op-geth execution node..."
 ./geth \
     --datadir="$GETH_DATA_DIR" \
     --verbosity="$VERBOSITY" \
-    --http \
-    --http.corsdomain="*" \
-    --http.vhosts="*" \
-    --http.addr=0.0.0.0 \
-    --http.port="$RPC_PORT" \
-    --http.api=web3,debug,eth,net,engine \
     --authrpc.addr=0.0.0.0 \
     --authrpc.port="$AUTHRPC_PORT" \
     --authrpc.vhosts="*" \
@@ -77,15 +72,21 @@ echoBanner "Starting node..."
     --ws.port="$WS_PORT" \
     --ws.origins="*" \
     --ws.api=debug,eth,net,engine \
+    --http \
+    --http.corsdomain="*" \
+    --http.vhosts="*" \
+    --http.addr=0.0.0.0 \
+    --http.port="$RPC_PORT" \
+    --http.api=web3,debug,eth,net,engine \
     --metrics \
     --metrics.addr=0.0.0.0 \
     --metrics.port="$METRICS_PORT" \
     --syncmode="$OP_GETH_SYNCMODE" \
     --gcmode="$OP_GETH_GCMODE" \
+    --port="$P2P_PORT" \
     --maxpeers=100 \
     --rollup.sequencerhttp="$SEQUENCER_HTTP" \
     --rollup.halt=major \
-    --port="$P2P_PORT" \
     --rollup.disabletxpoolgossip=true \
     $ADDITIONAL_ARGS # intentionally unquoted
 

diff --git a/op-node-entrypoint b/op-node-entrypoint
@@ -1,6 +1,12 @@
 #!/bin/bash
 set -eu
 
+echoBanner() {
+	echo -e "\n------------------------------------------------------------------------------------------------------------"
+	echo -e "$@"
+	echo -e "------------------------------------------------------------------------------------------------------------\n"
+}
+
 get_public_ip() {
   # Define a list of HTTP-based providers
   local PROVIDERS=(
@@ -48,7 +54,10 @@ else
 fi
 export OP_NODE_P2P_ADVERTISE_IP=$PUBLIC_IP
 
+# Set the start flags
 echo "$OP_NODE_L2_ENGINE_AUTH_RAW" > "$OP_NODE_L2_ENGINE_AUTH"
 
+# Start service
+echoBanner "Starting op-node..."
 exec ./op-node \
   --rollup.config=$OP_NODE_ROLLUP_CONFIG
diff --git a/geth/lisk-hotfix.patch → op-node-lisk-hotfix.patch b/geth/lisk-hotfix.patch → op-node-lisk-hotfix.patch
diff --git a/reth/Dockerfile b/reth/Dockerfile
@@ -9,26 +9,34 @@ RUN git clone $REPO --branch op-node/$VERSION --single-branch . && \
     git switch -c branch-$VERSION && \
     bash -c '[ "$(git rev-parse HEAD)" = "$COMMIT" ]'
 
-RUN cd op-node && \
+# Temporary patch until our RaaS provider re-deploys the SystemConfig contract
+COPY op-node-lisk-hotfix.patch .
+RUN git apply op-node-lisk-hotfix.patch && \
+    cd op-node && \
     make VERSION=$VERSION op-node
 
 FROM rust:1.79 AS reth
 
 ARG FEATURES=jemalloc,asm-keccak,optimism
+ARG PROFILE=release
 
 WORKDIR /app
 
 RUN apt-get update && apt-get -y upgrade && apt-get install -y git libclang-dev pkg-config curl build-essential
 
-#TODO: Revert to upstream, once https://github.com/paradigmxyz/reth/pull/9105 is merged.
-ENV REPO=https://github.com/danyalprout/reth.git
-ENV COMMIT=c79cbf4a918480556bd37204942a2d69827fed9f
-RUN git clone $REPO . && git checkout $COMMIT
+ENV REPO=https://github.com/paradigmxyz/reth.git
+ENV VERSION=v1.0.3
+ENV COMMIT=390f30aadebcdd509e72cc04327c3b854de076a6
+RUN git clone $REPO --branch $VERSION --single-branch . && \
+    git switch -c branch-$VERSION && \
+    bash -c '[ "$(git rev-parse HEAD)" = "$COMMIT" ]'
 
-RUN cargo build --bin op-reth --locked --features $FEATURES --profile release
+RUN cargo build --bin op-reth --locked --features $FEATURES --profile $PROFILE
 
 FROM ubuntu:22.04
 
+ARG PROFILE=release
+
 RUN apt-get update && \
     apt-get install -y jq curl supervisor && \
     rm -rf /var/lib/apt/lists
@@ -37,11 +45,11 @@ RUN mkdir -p /var/log/supervisor
 WORKDIR /app
 
 COPY --from=op /app/op-node/bin/op-node ./
-COPY --from=reth /app/target/release/op-reth ./
+COPY --from=reth /app/target/$PROFILE/op-reth ./
 COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf
 COPY ./reth/reth-entrypoint ./execution-entrypoint
 COPY op-node-entrypoint .
 COPY sepolia ./sepolia
 COPY mainnet ./mainnet
 
-CMD ["/usr/bin/supervisord"]
+CMD ["/usr/bin/supervisord"]