Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: compact blocks prototype #3713

Draft
wants to merge 66 commits into
base: main
Choose a base branch
from
Draft
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
959a4b8
use celestia-core branch with compact blocks
cmwaters Jul 19, 2024
79cbec4
modify two node simple benchmark
cmwaters Jul 19, 2024
b9408ed
Merge branch 'main' into cal/compact-blocks
cmwaters Jul 19, 2024
6f5018e
update go mod
cmwaters Jul 22, 2024
0d92d07
start the tx client later
cmwaters Jul 23, 2024
fe475f1
make some modifications to e2e simple
cmwaters Jul 23, 2024
1584726
use latest compact blocks changes
cmwaters Jul 24, 2024
dd0c28d
Merge branch 'main' into cal/compact-blocks
cmwaters Jul 24, 2024
a7ea55b
add docker container version
cmwaters Jul 24, 2024
12d9412
bump go mod
cmwaters Jul 29, 2024
5b4d614
udpate metric gathering
cmwaters Jul 30, 2024
28f3f03
fix metrics
cmwaters Jul 30, 2024
3cb2353
panic in ante handler with 0 version
cmwaters Jul 30, 2024
9bc97a2
checkpoint
cmwaters Jul 30, 2024
a8007d8
checkpoint
cmwaters Jul 30, 2024
e4563ba
checkpoint
cmwaters Jul 31, 2024
75e4ea6
checkpoint
cmwaters Jul 31, 2024
76c34d6
checkpoint
cmwaters Aug 1, 2024
bc97f50
checkpoint
cmwaters Aug 8, 2024
6f42cdf
Merge branch 'main' into cal/compact-blocks
cmwaters Aug 8, 2024
66b38d5
checkpoint
cmwaters Aug 9, 2024
0c3586b
checkpoint
cmwaters Aug 9, 2024
e1d7500
checkpoint
cmwaters Aug 9, 2024
729e0a1
checkpoint
cmwaters Aug 12, 2024
bbce586
checkpoint
cmwaters Aug 13, 2024
0ab0600
checkpoint
cmwaters Aug 13, 2024
c22c574
checkpoint
cmwaters Aug 13, 2024
6c85542
checkpoint
cmwaters Aug 14, 2024
b1018cd
Merge branch 'main' into cal/compact-blocks
cmwaters Aug 14, 2024
17c3665
checkpoint
cmwaters Aug 14, 2024
7e9dcd4
fix msg gatekeeper
cmwaters Aug 14, 2024
f4a6dbb
checkpoint
cmwaters Aug 14, 2024
a0e20e0
Merge branch 'main' into cal/compact-blocks
cmwaters Aug 14, 2024
a161691
checkpoint
cmwaters Aug 14, 2024
d57d1af
checkpoint
cmwaters Aug 15, 2024
87d3e21
checkpoint
cmwaters Aug 16, 2024
02b512f
checkpoint
cmwaters Aug 16, 2024
d6729fa
checkpoint
cmwaters Aug 20, 2024
4ab6269
Merge branch 'main' into cal/compact-blocks
cmwaters Aug 20, 2024
1c088cd
fix compile errors
cmwaters Aug 20, 2024
937d100
checkpoint
cmwaters Aug 22, 2024
f05744e
checkpoint
cmwaters Aug 26, 2024
44110b0
checkpoint
cmwaters Aug 26, 2024
d4ba7a0
checkpoint
cmwaters Aug 28, 2024
2dd6eff
Merge branch 'main' into cal/compact-blocks
cmwaters Aug 28, 2024
8a062d2
checkpoint
cmwaters Aug 30, 2024
754d16c
checkpoint
cmwaters Aug 30, 2024
beed88a
checkpoint
cmwaters Aug 30, 2024
116d63d
checkpoint
cmwaters Aug 30, 2024
a84eaf9
introduce option to allow txsim to ignore submission errors
cmwaters Aug 30, 2024
9d4cb4a
update txsim
cmwaters Sep 2, 2024
ba55c12
fix txsim cli
cmwaters Sep 2, 2024
791afc3
checkpoint
cmwaters Sep 3, 2024
6a6ea1f
checkpoint
cmwaters Sep 3, 2024
4858de1
checkpoint
cmwaters Sep 5, 2024
c57acbc
Merge branch 'main' into cal/compact-blocks
cmwaters Sep 5, 2024
d8ade3c
checkpoint
cmwaters Sep 5, 2024
f359e10
checkpoint
cmwaters Sep 6, 2024
7c52cbc
checkpoint
cmwaters Sep 6, 2024
66c2208
checkpoint
cmwaters Sep 6, 2024
2f05007
checkpoint
cmwaters Sep 6, 2024
43ecd7e
checkpoint
cmwaters Sep 9, 2024
d937bd8
update
cmwaters Oct 28, 2024
681e04d
Merge branch 'main' into cal/compact-blocks
cmwaters Nov 1, 2024
86d3e0d
Merge branch 'main' into cal/compact-blocks
cmwaters Nov 1, 2024
0e61251
converge a little closer to main
cmwaters Nov 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
checkpoint
cmwaters committed Aug 30, 2024

Unverified

This commit is not signed, but one or more authors requires that any commit attributed to them is signed.
commit beed88a80fecb58e3f50740947e1325306c43933
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -253,5 +253,5 @@ replace (
github.com/gogo/protobuf => github.com/regen-network/protobuf v1.3.3-alpha.regen.1
github.com/syndtr/goleveldb => github.com/syndtr/goleveldb v1.0.1-0.20210819022825-2ae1ddf74ef7
//v1.39.0-tm-v0.34.29
github.com/tendermint/tendermint => github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830134440-cf354c5176b3
github.com/tendermint/tendermint => github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830140809-dddd61e91665
)
4 changes: 2 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
@@ -318,8 +318,8 @@ github.com/celestiaorg/bittwister v0.0.0-20231213180407-65cdbaf5b8c7 h1:nxplQi8w
github.com/celestiaorg/bittwister v0.0.0-20231213180407-65cdbaf5b8c7/go.mod h1:1EF5MfOxVf0WC51Gb7pJ6bcZxnXKNAf9pqWtjgPBAYc=
github.com/celestiaorg/blobstream-contracts/v3 v3.1.0 h1:h1Y4V3EMQ2mFmNtWt2sIhZIuyASInj1a9ExI8xOsTOw=
github.com/celestiaorg/blobstream-contracts/v3 v3.1.0/go.mod h1:x4DKyfKOSv1ZJM9NwV+Pw01kH2CD7N5zTFclXIVJ6GQ=
github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830134440-cf354c5176b3 h1:uQpyfmqWO06uHv0x31z9kLZx/xhyqTfQ1lziY60vmYw=
github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830134440-cf354c5176b3/go.mod h1:gr8NlCcouMlH2bPBHySRXRXlTEZWLw8UpFyY0seMpR0=
github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830140809-dddd61e91665 h1:VGNNKqJttVDL8hwVAxp8bZllyVIIDWWVx82tj4RTNWQ=
github.com/celestiaorg/celestia-core v1.38.0-tm-v0.34.29.0.20240830140809-dddd61e91665/go.mod h1:gr8NlCcouMlH2bPBHySRXRXlTEZWLw8UpFyY0seMpR0=
github.com/celestiaorg/cosmos-sdk v1.24.1-sdk-v0.46.16 h1:SeQ7Y/CyOcUMKo7mQiexaj/pZ/xIgyuZFIwYZwpSkWE=
github.com/celestiaorg/cosmos-sdk v1.24.1-sdk-v0.46.16/go.mod h1:Bpl1LSWiDpQumgOhhMTZBMopqa0j7fRasIhvTZB44P0=
github.com/celestiaorg/go-square/v2 v2.0.0-rc2 h1:4D+ASgZGYVCsffc2uhPagACrvNiLZu9/CqNYvnlHCgg=
4 changes: 2 additions & 2 deletions test/e2e/experiment/compact_blocks/main.go
Original file line number Diff line number Diff line change
@@ -18,7 +18,7 @@
)

const (
compactBlocksVersion = "4021c9f" //"a28b9e7"
compactBlocksVersion = "986c330" //"a28b9e7"

Check failure on line 21 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

commentFormatting: put a space between `//` and comment text (gocritic)
)

func main() {
@@ -72,7 +72,7 @@
"128000-128000",
1,
testnet.DefaultResources,
gRPCEndpoints[2:6],
gRPCEndpoints[2:7],
)
if err != nil {
return err
@@ -100,13 +100,13 @@
}
log.Print("Setting up trace push config")
for _, node := range network.Nodes() {
if err = node.Instance.SetEnvironmentVariable(trace.PushBucketName, pushConfig.BucketName); err != nil {

Check failure on line 103 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

SA1019: node.Instance.SetEnvironmentVariable is deprecated: Use the new package knuu instead. (staticcheck)
return fmt.Errorf("failed to set TRACE_PUSH_BUCKET_NAME: %v", err)
}
if err = node.Instance.SetEnvironmentVariable(trace.PushRegion, pushConfig.Region); err != nil {

Check failure on line 106 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

SA1019: node.Instance.SetEnvironmentVariable is deprecated: Use the new package knuu instead. (staticcheck)
return fmt.Errorf("failed to set TRACE_PUSH_REGION: %v", err)
}
if err = node.Instance.SetEnvironmentVariable(trace.PushAccessKey, pushConfig.AccessKey); err != nil {

Check failure on line 109 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

SA1019: node.Instance.SetEnvironmentVariable is deprecated: Use the new package knuu instead. (staticcheck)
return fmt.Errorf("failed to set TRACE_PUSH_ACCESS_KEY: %v", err)
}
if err = node.Instance.SetEnvironmentVariable(trace.PushKey, pushConfig.SecretKey); err != nil {
@@ -124,7 +124,7 @@
}

for _, node := range network.Nodes() {
if err = node.Instance.SetLatencyAndJitter(40, 10); err != nil {

Check failure on line 127 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

SA1019: node.Instance.SetLatencyAndJitter is deprecated: Use the new package knuu instead. (staticcheck)
return fmt.Errorf("failed to set latency and jitter: %v", err)
}
}
@@ -155,7 +155,7 @@
log.Printf("Height: %v", status.SyncInfo.LatestBlockHeight)

case <-timeout.C:
network.StopTxClients()

Check failure on line 158 in test/e2e/experiment/compact_blocks/main.go

GitHub Actions / lint / golangci-lint

Error return value of `network.StopTxClients` is not checked (errcheck)
log.Println("--- COLLECTING DATA")
file := "/Users/callum/Developer/go/src/github.com/celestiaorg/big-blocks-research/traces"
if err := trace.S3Download(file, network.ChainID(), pushConfig, schema.RoundStateTable, schema.BlockTable, schema.ProposalTable, schema.CompactBlockTable, schema.MempoolRecoveryTable); err != nil {

Unchanged files with check annotations Beta

txClients []*TxSim
}
func New(name string, seed int64, grafana *GrafanaInfo, chainID string,

Check failure on line 30 in test/e2e/testnet/testnet.go

GitHub Actions / lint / golangci-lint

unused-parameter: parameter 'chainID' seems to be unused, consider removing or renaming it as _ (revive)
genesisModifiers ...genesis.Modifier) (
*Testnet, error,
) {