Geth v1.13.2 #1116

darioush · 2024-03-06T02:04:39Z

Why this should be merged

#1112

How this works

Takes code from upstream

How this was tested

[x] CI
[ ] Testnet

How is this documented

TODO: update deprecated flag

* all: implement path-based state scheme * all: edits from review * core/rawdb, trie/triedb/pathdb: review changes * core, light, trie, eth, tests: reimplement pbss history * core, trie/triedb/pathdb: track block number in state history * trie/triedb/pathdb: add history documentation * core, trie/triedb/pathdb: address comments from Peter's review Important changes to list: - Cache trie nodes by path in clean cache - Remove root->id mappings when history is truncated * trie/triedb/pathdb: fallback to disk if unexpect node in clean cache * core/rawdb: fix tests * trie/triedb/pathdb: rename metrics, change clean cache key * trie/triedb: manage the clean cache inside of disk layer * trie/triedb/pathdb: move journal function * trie/triedb/path: fix tests * trie/triedb/pathdb: fix journal * trie/triedb/pathdb: fix history * trie/triedb/pathdb: try to fix tests on windows * core, trie: address comments * trie/triedb/pathdb: fix test issues --------- Co-authored-by: Felix Lange <[email protected]> Co-authored-by: Martin Holst Swende <[email protected]>

Co-authored-by: Felix Lange <[email protected]>

…27842)

This change rearranges the accessor methods in block.go and fixes some minor issues with the copy-on-write logic of block data. Fixed issues: - Block.WithWithdrawals did not create a shallow copy of the block. - Block.WithBody copied the header unnecessarily, and did not preserve withdrawals. However, the bugs did not affect any code in go-ethereum because blocks are *always* created using NewBlockWithHeader().WithBody().WithWithdrawals()

This adds support for the "yParity" field in transaction objects returned by RPC APIs. We somehow forgot to add this field even though it has been in the spec for a long time.

Co-authored-by: Felix Lange <[email protected]>

…752)

Fixes a graphql-dos --------- Co-authored-by: Sina Mahmoodi <[email protected]> Co-authored-by: Sina Mahmoodi <[email protected]>

Moving the response sending there allows tracking all peer goroutines in the peer WaitGroup.

Co-authored-by: Felix Lange <[email protected]>

…(#27853) Context: The UpdateContractCode method was introduced for the state storage commitment schemes that include the whole code for their commitment computation. It must therefore be called before the root hash is computed at the end of IntermediateRoot. This should have no impact on the MPT since, in this context, the method is a no-op.

… (#27891) This change removes a chainconfig parameter passed into rawdb.ReadLogs, which is not used nor needed. It also modifies the filter loop slightly, avoiding a labeled break and instead using a method. This change does not modify any behaviour.

build(deps): bump github.com/supranational/blst Bumps [github.com/supranational/blst](https://github.com/supranational/blst) from 0.3.11-0.20230406105308-e9dfc5ee724b to 0.3.11. - [Release notes](https://github.com/supranational/blst/releases) - [Commits](https://github.com/supranational/blst/commits/v0.3.11) --- updated-dependencies: - dependency-name: github.com/supranational/blst dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Block takes a number and a hash. The spec is unclear on what should happen in this case, leaving it an implemenation detail. With this change, we return an error in case both number and hash are passed in.

* all: activate pbss * core/rawdb: fix compilation error * cma, core, eth, les, trie: address comments * cmd, core, eth, trie: polish code * core, cmd, eth: address comments * cmd, core, eth, les, light, tests: address comment * cmd/utils: shorten log message * trie/triedb/pathdb: limit node buffer size to 1gb * cmd/utils: fix opening non-existing db * cmd/utils: rename flag name * cmd, core: group chain history flags and fix tests * core, eth, trie: fix memory leak in snapshot generation * cmd, eth, internal: deprecate flags * all: enable state tests for pathdb, fixes * cmd, core: polish code * trie/triedb/pathdb: limit the node buffer size to 256mb --------- Co-authored-by: Martin Holst Swende <[email protected]> Co-authored-by: Péter Szilágyi <[email protected]>

This should fix #27726. With enough load, it might happen that the SetPongHandler callback gets invoked before the call to SetReadDeadline is made in pingLoop. When this occurs, the socket will end up with a 30s read deadline even though it got the pong, which will lead to a timeout. The fix here is processing the pong on pingLoop, synchronizing with the code that sends the ping.

…3.2-x

darioush · 2024-03-06T23:07:58Z

core/blockchain.go

@@ -464,6 +506,17 @@ func (bc *BlockChain) dispatchTxUnindexer() {
 		return
 	}
 	defer sub.Unsubscribe()
+	log.Info("Initialized transaction unindexer", "limit", txLookupLimit)
+
+	// XXX


not sure why the test fails when I uncomment the below.
Not sure we need to take this change either.

There is an issue with unindexer currently, and applying this change can trigger the issue. IMO we should fix this first, gracefully handle the situation and then apply this so we can unindex the txs on startup.

core/blockchain.go

darioush · 2024-03-06T23:09:55Z

trie/triedb/pathdb/layertree.go

@@ -142,7 +142,32 @@ func (tree *layerTree) cap(root common.Hash, layers int) error {
 			return err
 		}
 		// Replace the entire layer tree with the flat base
-		tree.layers = map[common.Hash]layer{base.rootHash(): base}
+		// tree.layers = map[common.Hash]layer{base.rootHash(): base}
+		// XXX


Seems geth adopts a different semantics for commit than what we rely on in blockchain.go
I implemented this version which allows uncommitted layers to remain on top instead of flattening everything.

darioush · 2024-03-06T23:12:33Z

miner/worker.go

+	mu         sync.RWMutex   // The lock used to protect the coinbase and extra fields
+	coinbase   common.Address
+	clock      *mockable.Clock // Allows us mock the clock for testing
+	beaconRoot *common.Hash    // XXX: this is not set anywhere


I think we should support setting beaconRoot somehow during block building, but not sure what we would put in it.

see comment in types/block.go

metrics/prometheus/prometheus.go

core/state/statedb.go

darioush · 2024-03-06T23:21:31Z

core/blockchain_repair_test.go

 			b.SetCoinbase(common.Address{0x01})
+			tx, err := types.SignTx(types.NewTransaction(b.TxNonce(addr1), common.Address{0x01}, big.NewInt(10000), params.TxGas, common.Big1, nil), signer, key1)
+			require.NoError(err)
+			b.AddTx(tx)


Had to make these changes since in geth, the roots are different because of the coinbase payment.
Here we just keep on with the empty trie and that doesn't work for the pathdb (doesn't expect repeated roots)

core/rawdb/database.go

core/types/block.go

ceyonur · 2024-03-20T11:16:08Z

core/blockchain.go

@@ -464,6 +506,17 @@ func (bc *BlockChain) dispatchTxUnindexer() {
 		return
 	}
 	defer sub.Unsubscribe()
+	log.Info("Initialized transaction unindexer", "limit", txLookupLimit)
+
+	// XXX


There is an issue with unindexer currently, and applying this change can trigger the issue. IMO we should fix this first, gracefully handle the situation and then apply this so we can unindex the txs on startup.

eth/ethconfig/config.go

core/state/statedb.go

core/blockchain.go

metrics/prometheus/prometheus.go

ceyonur · 2024-03-20T12:29:08Z

miner/worker.go

+	mu         sync.RWMutex   // The lock used to protect the coinbase and extra fields
+	coinbase   common.Address
+	clock      *mockable.Clock // Allows us mock the clock for testing
+	beaconRoot *common.Hash    // XXX: this is not set anywhere


see comment in types/block.go

…3.2-x

trie/triedb/hashdb/database.go

…3.2-x

* add log formats from upstream * remove align

darioush · 2024-04-02T16:22:49Z

replaced by #1141

ucwong and others added 30 commits August 1, 2023 14:12

cmd/bootnode: fix timer leak (#27754)

9d744f0

cmd/geth: disable automaxprocs log (#27814)

ff97b4c

Co-authored-by: Felix Lange <[email protected]>

p2p: use atomic types (#27764)

d04bde0

Co-authored-by: Felix Lange <[email protected]>

build: upgrade -dlgo version to Go 1.20.7 (#27835)

8224bb9

eth/protocols/snap: fix batch writer when resuming an aborted sync (#…

6e934f4

…27842)

all: remove trailing whitespace (#27741)

57cdbae

core/types: support yParity field in JSON transactions (#27744)

bb148dd

This adds support for the "yParity" field in transaction objects returned by RPC APIs. We somehow forgot to add this field even though it has been in the spec for a long time.

log: use atomic types (#27763)

5c30541

Co-authored-by: Felix Lange <[email protected]>

internal/ethapi, graphql: correct comments about gas price logic (#27…

60070fe

…752)

internal: use atomic type (#27858)

8574767

accounts: use atomic type (#27857)

d14c07d

internal/ethapi: add state override to estimateGas (#27845)

eeebb07

p2p/simulations/examples: use atomic.Int64 (#27861)

6d2bcb9

graphql: avoid greedy allocation (#27873)

0d772b9

Fixes a graphql-dos --------- Co-authored-by: Sina Mahmoodi <[email protected]> Co-authored-by: Sina Mahmoodi <[email protected]>

core/vm: update 4844 - point evaluation precompile address (#27874)

e13fa32

p2p: move ping handling into pingLoop goroutine (#27887)

7ec60d5

Moving the response sending there allows tracking all peer goroutines in the peer WaitGroup.

graphql: add yParity field for transactions (#27882)

10d9f93

Co-authored-by: Felix Lange <[email protected]>

params: release go-ethereum v1.12.1

9c216bd

params: begin v1.12.2 release cycle

d9a8b0f

graphql: validate block params (#27876)

5e89ff4

Block takes a number and a hash. The spec is unclear on what should happen in this case, leaving it an implemenation detail. With this change, we return an error in case both number and hash are passed in.

params: switch release family to 1.13 to prep for PBSS

e2507a1

core/txpool/blobpool: fix metrics name for prometheus export (#27901)

6ddb92c

ethdb/leveldb: support more than 7 levels in metrics (#27904)

35f7f3d

darioush changed the base branch from geth-v1.12.2-x to master March 6, 2024 22:21

darioush added 4 commits March 6, 2024 14:33

Merge branch 'master' of github.com:ava-labs/subnet-evm into geth-v.1…

80fdc34

…3.2-x

unused files

9b482e6

lint

a2a2b35

resolve some of the TODOS

e44d623

darioush commented Mar 6, 2024

View reviewed changes

core/blockchain.go Outdated Show resolved Hide resolved

darioush commented Mar 6, 2024

View reviewed changes

metrics/prometheus/prometheus.go Outdated Show resolved Hide resolved

darioush commented Mar 6, 2024

View reviewed changes

core/state/statedb.go Outdated Show resolved Hide resolved

darioush commented Mar 6, 2024

View reviewed changes

ceyonur reviewed Mar 20, 2024

View reviewed changes

darioush added 6 commits March 20, 2024 14:40

remove chaininfo metric

f38bab8

update schema accounting (database.go)

52952b5

deprecate tx-lookup-limit

95120b7

fix testDeleteStorage

1c2e8f9

Merge branch 'master' of github.com:ava-labs/subnet-evm into geth-v.1…

8c0c7bd

…3.2-x

fix race

2e2beea

darioush changed the title ~~Geth v.13.2 x~~ Geth v1.13.2 Mar 21, 2024

ceyonur reviewed Mar 22, 2024

View reviewed changes

trie/triedb/hashdb/database.go Outdated Show resolved Hide resolved

darioush and others added 7 commits March 25, 2024 14:59

triedb/pathdb: document deviation from upstream

4af41a1

worker.go: update TODO comments

cadd7bb

Merge branch 'master' of github.com:ava-labs/subnet-evm into geth-v.1…

1283cb0

…3.2-x

fix coreth versioning

02377fb

add log formats from upstream (#1140)

4bdba28

* add log formats from upstream * remove align

Merge branch 'master' into geth-v.13.2-x

a3baeec

add comment

eb9d16a

darioush closed this Apr 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Geth v1.13.2 #1116

Geth v1.13.2 #1116

darioush commented Mar 6, 2024 •

edited

Loading

darioush Mar 6, 2024

ceyonur Mar 20, 2024

darioush Mar 6, 2024

darioush Mar 6, 2024

ceyonur Mar 20, 2024

darioush Mar 6, 2024

ceyonur Mar 20, 2024

ceyonur Mar 20, 2024

darioush commented Apr 2, 2024

Geth v1.13.2 #1116

Geth v1.13.2 #1116

Conversation

darioush commented Mar 6, 2024 • edited Loading

Why this should be merged

How this works

How this was tested

How is this documented

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

darioush commented Apr 2, 2024

darioush commented Mar 6, 2024 •

edited

Loading