Update celo blockchain v1.6.1 (#192)

Updates the celo-blockchain dependency to v1.6.1 and additionally:

Upgrades kliento to match the new celo-blockchain dependency.

Upgrades golangci-lint to a working version (the existing version was facing OOM errors),
some code needed to be reformatted to conform with the new version.

Expands online CLI help, simplifies README.md.

Improves error reporting on startup.

Removes some unused code.

Fixes path for static nodes, when geth.staticnodes flag was set, rosetta
was not generating the static nodes file in the correct directory.

Ensure celo node indexes transactions forever, previously the associated celo-blockchain
node would only keep transactions indexed by their hashes for roughly 4 months.

.gitignore

@@ -5,4 +5,5 @@ main
 .DS_Store
 bin/
 celo/
-*.tar.gz
+*.tar.gz
+.vscode

.golangci.yml

@@ -10,13 +10,11 @@ run:
 linters:
   disable-all: true
   enable:
-    - deadcode
     - errcheck
     - govet
     - ineffassign
     - staticcheck
     - unused
-    - varcheck
 
     - goconst
     - goimports
@@ -41,4 +39,4 @@ issues:
         - misspell
 
   exclude:
-    - composites
+    - composites

Dockerfile

@@ -43,7 +43,7 @@ RUN go build --tags musl -o rosetta .
 # Outputs: rosetta & geth binaries on /usr/loca/bin
 #---------------------------------------------------------------------
 # geth mainnet (1.5.1)
-FROM us.gcr.io/celo-org/geth:1.5.4
+FROM us.gcr.io/celo-org/geth:1.6.1
 ARG COMMIT_SHA
 
 RUN apk add --no-cache ca-certificates

Makefile

@@ -10,7 +10,7 @@ GO ?= latest
 GITHUB_ORG?=celo-org
 GITHUB_REPO?=rosetta
 
-GOLANGCI_VERSION=1.32.2
+GOLANGCI_VERSION=1.50.0
 GOLANGCI_exists := $(shell command -v golangci-lint 2> /dev/null)
 GOLANGCI_v_installed := $(shell echo $(shell golangci-lint --version) | cut -d " " -f 4)
 

README.md

@@ -40,35 +40,7 @@ For an understanding of inputs & outputs check [servicer.go](./service/rpc/servi
 
 ## Command line arguments
 
-The main command is `rosetta run`, whose arguments are:
-
-```txt
-Usage:
-  rosetta run [flags]
-
-Flags:
-      --datadir string            datadir to use
-      --geth.binary string        Path to the celo-blockchain binary
-      --geth.bootnodes string     Bootnodes to use (separated by ,)
-      --geth.cache string         Memory (in MB) allocated to geth's internal caching
-      --geth.gcmode string        Geth garbage collection mode (full, archive) (default "full")
-      --geth.genesis string       (Optional) path to the genesis.json, for use with custom chains
-      --geth.ipcpath string       Path to the geth ipc file
-      --geth.logfile string       Path to logs file
-      --geth.maxpeers string      Maximum number of network peers (network disabled if set to 0) (default: 1100) (default "1100")
-      --geth.network string       Network to use, either 'mainnet', 'alfajores', or 'baklava'
-      --geth.publicip string      Public Ip to configure geth (sometimes required for discovery)
-      --geth.rpcaddr string       Geth HTTP-RPC server listening interface (default "127.0.0.1")
-      --geth.rpcport string       Geth HTTP-RPC server listening port (default "8545")
-      --geth.rpcvhosts string     Geth comma separated list of virtual hostnames from which to accept requests (default "localhost")
-      --geth.staticnodes string   StaticNode to use (separated by ,)
-      --geth.syncmode string      Geth blockchain sync mode (fast, full, light) (default "fast")
-      --geth.verbosity string     Geth log verbosity (number between [1-5])
-  -h, --help                      help for run
-      --rpc.address string        Listening address for http server
-      --rpc.port uint             Listening port for http server (default 8080)
-      --rpc.reqTimeout duration   Timeout for requests to this service, this also controls the timeout sent to the blockchain node for trace transaction requests (default 2m0s)
-```
+Arguments are described in the help output of the CLI.
 
 Every argument can be defined using environment variables using `ROSETTA_` prefix; and replacing `.` for `_`; for example:
 
@@ -77,11 +49,9 @@ ROSETTA_DATADIR="/my/dir"
 ROSETTA_GETH_NETWORK="alfajores"
 ```
 
-Note that from Rosetta `v0.8.4` onwards, it is no longer necessary to pass in either `--geth.bootnodes` or `--geth.staticnodes`, as the geth flag `--alfajores`, `--baklava`, or no flag (for mainnet) will be set automatically, which sets the geth bootnodes appropriately. These flags may still optionally be used but are not recommended if there is not a specific reason to do so.
-
 ## Running the Rosetta RPC Server
 
-Running the Rosetta RPC Server from scratch will take some time to sync, since it runs a full archive node in the background. While it may be possible to run the Construction API in the future with a non-archive node, this is still required by the Rosetta spec for the Data API implementation in order to perform balance reconciliation.
+Running the Rosetta RPC Server from scratch will take a very long time to sync and require at least 1.5TB of storage space, since it runs a full archive node in the background. While it may be possible to run the Construction API in the future with a non-archive node, this is still required by the Rosetta spec for the Data API implementation in order to perform balance reconciliation.
 
 ### Version 1: Running from `rosetta` source code
 
@@ -211,6 +181,14 @@ Rosetta uses [kliento](https://github.com/celo-org/kliento) to interact with the
 
 ## How to run rosetta-cli-checks
 
+_Note that running these checks is most likely infeasible for most people on
+mainnet because you will need at least 1.5 terrabytes of space and several days
+to be able to sync the chain. Under the hood, the service is syncing a full
+archive node, which takes a long time. The construction service needs to reach
+the tip before submitting transactions. The data checks will take a while to
+complete as well (likely a couple of days on a normal laptop with the current
+settings) as they reconcile balances for the entire chain._
+
 - Install the [`rosetta-cli`](https://github.com/coinbase/rosetta-cli) according to the instructions. (Note that on Mac, installing the `rosetta-cli` to `/usr/local/bin` or adding its location to you `$PATH` will allow you to call `rosetta-cli` directly on the command line rather than needing to provide the path to the executable). Current testing has been done with `v0.5.16` of the `rosetta-cli`.
 - Run the Rosetta service in the background for the respective network (currently only alfajores for both Data and Construction checks)
 - Run the CLI checks for alfajores as follows:
@@ -220,7 +198,6 @@ Rosetta uses [kliento](https://github.com/celo-org/kliento) to interact with the
 rosetta-cli check:construction --configuration-file PATH/TO/rosetta/rosetta-cli-conf/testnet/cli-config.json
 ```
 
-_Note that running the checks to completion will take a long time if this is the first time you are running Rosetta locally. Under the hood, the service is syncing a full archive node, which takes time (likely a couple of days on a normal laptop). The construction service needs to reach the tip before submitting transactions. The data checks will take a while to complete as well (likely a couple of days on a normal laptop with the current settings) as they reconcile balances for the entire chain._
 
 ### How to generate `bootstrap_balances.json`
 

analyzer/operations.go

@@ -198,17 +198,20 @@ func NewAuthorizeSigner(from common.Address, signer common.Address, authorizeOp
 
 // Ex. Slash(penalty=110, reward=100 cGlD), tobinTax == 10%
 // Transfer Operation:
-// 	lockedGoldContractAccMain        -9
-// 	communityFundAccMain              9
-// 	lockedGoldContractAccMain        -1
-// 	tobinRecipientAccMain             1
+//
+//	lockedGoldContractAccMain        -9
+//	communityFundAccMain              9
+//	lockedGoldContractAccMain        -1
+//	tobinRecipientAccMain             1
+//
 // LockedGold Operation (created from `AccountSlashed(slashed, -100, reporter, 110)` event):
-// 	slashedAccLockedGoldNonVoting    -110
-// 	reporterAccLockedGoldNonVoting    100
-// 	lockedGoldContractAccMain        -9
-// 	communityFundAccMain              9
-// 	lockedGoldContractAccMain        -1
-// 	tobinRecipientAccMain             1
+//
+//	slashedAccLockedGoldNonVoting    -110
+//	reporterAccLockedGoldNonVoting    100
+//	lockedGoldContractAccMain        -9
+//	communityFundAccMain              9
+//	lockedGoldContractAccMain        -1
+//	tobinRecipientAccMain             1
 func NewSlash(slashed, slasher, communityFund, lockedGoldAddr common.Address, penalty, reward *big.Int, tobinTax *TobinTax) *Operation {
 	communityFundReward := new(big.Int).Sub(penalty, reward)
 	return &Operation{
@@ -223,20 +226,25 @@ func NewSlash(slashed, slasher, communityFund, lockedGoldAddr common.Address, pe
 
 // Ex. lock(100 cGlD), tobinTax == 10%
 // Transfer Operation:
-// 	fromAccMain         	 	-90
-// 	lockedGoldContractAccMain    90
-// 	fromAccMain			    	-10
-// 	tobinRecipientAccMain        10
+//
+//	fromAccMain         	 	-90
+//	lockedGoldContractAccMain    90
+//	fromAccMain			    	-10
+//	tobinRecipientAccMain        10
+//
 // LockedGold Operation (created from `GoldLocked(fromAccount, 90)` event):
-// 	fromAccMain                 -90
-// 	lockedGoldContractAccMain    90
-//  fromAccLockedNonVoting       90
+//
+//		fromAccMain                 -90
+//		lockedGoldContractAccMain    90
+//	 fromAccLockedNonVoting       90
+//
 // Reconciled Operation:
-// 	fromAccMain         	 	-90
-// 	lockedGoldContractAccMain    90
-// 	fromAccMain			    	-10
-// 	tobinRecipientAccMain        10
-//  fromAccLockedNonVoting       90
+//
+//		fromAccMain         	 	-90
+//		lockedGoldContractAccMain    90
+//		fromAccMain			    	-10
+//		tobinRecipientAccMain        10
+//	 fromAccLockedNonVoting       90
 func NewLockGold(addr, lockedGoldAddr common.Address, value *big.Int) *Operation {
 	return &Operation{
 		Type:       OpLockGold,
@@ -251,16 +259,19 @@ func NewLockGold(addr, lockedGoldAddr common.Address, value *big.Int) *Operation
 
 // Ex. Withdraw(100 cGlD), tobinTax == 10%
 // Transfer Operation:
-// 	lockedGoldContractAccMain   -90
-// 	toAccMain         	 	     90
-// 	lockedGoldContractAccMain   -10
-// 	tobinRecipientAccMain        10
+//
+//	lockedGoldContractAccMain   -90
+//	toAccMain         	 	     90
+//	lockedGoldContractAccMain   -10
+//	tobinRecipientAccMain        10
+//
 // LockedGold Operation (created from `GoldWithdraw(toAccount, 100)` event):
-// 	lockedGoldContractAccMain   -90
-// 	toAccMain         	 	     90
-// 	lockedGoldContractAccMain   -10
-// 	tobinRecipientAccMain        10
-//  AccLockedPending            -100
+//
+//		lockedGoldContractAccMain   -90
+//		toAccMain         	 	     90
+//		lockedGoldContractAccMain   -10
+//		tobinRecipientAccMain        10
+//	 AccLockedPending            -100
 func NewWithdrawGold(addr, lockedGoldAddr common.Address, value *big.Int, tobinTax *TobinTax) *Operation {
 	return &Operation{
 		Type:       OpWithdrawGold,
@@ -435,26 +446,32 @@ func ReconcileLogOpsWithTransfers(logOps, transferOps []Operation, tobinTax *Tob
 // by combining their balance changes
 //
 // operations match with tobin tax iff
-// 		logOp.Type == OpLockGold &&
-// 		logOp.lockedGoldContractAccMain.diff == trOp.lockedGoldContractAccMain.diff &&
-// 		logOp.fromAccMain.diff - trOp.fromAccMain.diff == trOp.tobinRecipientAccMain.diff
+//
+//	logOp.Type == OpLockGold &&
+//	logOp.lockedGoldContractAccMain.diff == trOp.lockedGoldContractAccMain.diff &&
+//	logOp.fromAccMain.diff - trOp.fromAccMain.diff == trOp.tobinRecipientAccMain.diff
 //
 // Ex. lock(100 cGlD), tobinTax == 10%
 // Transfer Operation:
-// 	fromAccMain         	 	-90
-// 	lockedGoldContractAccMain    90
-// 	fromAccMain			    	-10 // diff == -100
-// 	tobinRecipientAccMain        10
+//
+//	fromAccMain         	 	-90
+//	lockedGoldContractAccMain    90
+//	fromAccMain			    	-10 // diff == -100
+//	tobinRecipientAccMain        10
+//
 // LockedGold Operation (created from `GoldLocked(fromAccount, 90)` event):
-// 	fromAccMain                 -90 // diff == -90
-// 	lockedGoldContractAccMain    90
-//  fromAccLockedNonVoting       90
+//
+//		fromAccMain                 -90 // diff == -90
+//		lockedGoldContractAccMain    90
+//	 fromAccLockedNonVoting       90
+//
 // Reconciled Operation:
-// 	fromAccMain         	 	-90
-// 	lockedGoldContractAccMain    90
-// 	fromAccMain			    	-10
-// 	tobinRecipientAccMain        10
-//  fromAccLockedNonVoting       90
+//
+//		fromAccMain         	 	-90
+//		lockedGoldContractAccMain    90
+//		fromAccMain			    	-10
+//		tobinRecipientAccMain        10
+//	 fromAccLockedNonVoting       90
 func MatchAndReconcileLogOpWithTransfer(logOp, trOp *Operation, tobinTax *TobinTax, lockedGold common.Address) bool {
 
 	// If the operations match as is then we can just return true

cmd/cli/block.go

@@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
-    http://www.apache.org/licenses/LICENSE-2.0
+	http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,

cmd/cli/cli.go

@@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
-    http://www.apache.org/licenses/LICENSE-2.0
+	http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,

cmd/cli/reconcile.go

@@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
-    http://www.apache.org/licenses/LICENSE-2.0
+	http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,

cmd/root.go

@@ -5,7 +5,7 @@ Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.
 You may obtain a copy of the License at
 
-    http://www.apache.org/licenses/LICENSE-2.0
+	http://www.apache.org/licenses/LICENSE-2.0
 
 Unless required by applicable law or agreed to in writing, software
 distributed under the License is distributed on an "AS IS" BASIS,