Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(zk_toolbox): add multi-chain CI integration test #2594

Merged
merged 165 commits into from
Aug 30, 2024
Merged
Show file tree
Hide file tree
Changes from 164 commits
Commits
Show all changes
165 commits
Select commit Hold shift + click to select a range
1b74492
feat: add option to apply a port offset on chain initialization
manuelmauro Aug 5, 2024
e9dba06
fix: fix help message for port-offset option
manuelmauro Aug 5, 2024
9b648e5
docs: add note on how to deal with port offset when initializing ecos…
manuelmauro Aug 5, 2024
f720090
fix: fix url scheme for web sockets
manuelmauro Aug 6, 2024
c30b46d
fix: use correct property for ws_url
manuelmauro Aug 6, 2024
531afaf
test: do not make chain_rollup the default chain in CI
manuelmauro Aug 6, 2024
ea61f52
fix: add mandatory option to chain create
manuelmauro Aug 6, 2024
a2f0d8b
fix: assign correct chain to environment variable
manuelmauro Aug 7, 2024
46a8df3
fix: use correct chain in recovery tests
manuelmauro Aug 7, 2024
4baf084
fix: use correct chain in revert tests
manuelmauro Aug 7, 2024
8a84bc1
fix: use correct chain in upgrade test
manuelmauro Aug 7, 2024
2b9d147
fix: take a snapshot of the right chain
manuelmauro Aug 7, 2024
5d264fa
fix: use correct chain in snapshot creation
manuelmauro Aug 7, 2024
cd937fb
feat: add `zk_supervisor test all` subbommand
manuelmauro Aug 8, 2024
2ea236b
refactor: use `zk_supervisor test all` in CI
manuelmauro Aug 8, 2024
ead8204
fix: run external node rather than server
manuelmauro Aug 8, 2024
86ed38a
refactor: remove ext node config and init from `zk_supervisor test al…
manuelmauro Aug 8, 2024
8bb9555
style: unused vars
manuelmauro Aug 8, 2024
86a75ca
test: fail fast
manuelmauro Aug 8, 2024
8e794ea
fix: clippy
manuelmauro Aug 8, 2024
395adcf
ci: add test steps for Validium chain
manuelmauro Aug 8, 2024
468e43e
fix: specify port offset in validium chain
manuelmauro Aug 8, 2024
ef34819
ci: run tests concurrently
manuelmauro Aug 9, 2024
eff3e12
ci: wait for tests to run
manuelmauro Aug 9, 2024
1d16dfa
fix: concurrently run tests on multiple chains
manuelmauro Aug 9, 2024
867c06a
feat: make build deps step optional in tests
manuelmauro Aug 9, 2024
8f1969a
fix: build deps in CI
manuelmauro Aug 9, 2024
832ed36
ci: run serve in CI rather than in `zk_supervisor test all` subcommand
manuelmauro Aug 12, 2024
25bbd81
fix: add port-offset to more services
manuelmauro Aug 12, 2024
1d82828
fix: fix typo when checking for new --no-deps flag
manuelmauro Aug 12, 2024
ea4dd65
feat: add current_chain method to EcosystemConfig
manuelmauro Aug 12, 2024
c61609c
fix: clone origina shell instead of creating a new one
manuelmauro Aug 12, 2024
bb4a83b
test: run revert tests only once (with EN)
manuelmauro Aug 12, 2024
5ca9cf1
refactor: use update_ports in apply_ports_offset
manuelmauro Aug 12, 2024
4c79dcb
refactor: lint
manuelmauro Aug 12, 2024
2c29783
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 12, 2024
866003a
refactor: lint more
manuelmauro Aug 12, 2024
e529941
ci: log ports
manuelmauro Aug 12, 2024
fb4b050
ci: use netstat instead of lsof
manuelmauro Aug 12, 2024
84c23a8
ci: run netstat with super user privileges
manuelmauro Aug 12, 2024
b83961c
feat: use chain ID in port_offset during initialization
manuelmauro Aug 12, 2024
cb2e5f4
fix: update contract_verifier port on update_ports
manuelmauro Aug 13, 2024
310ef67
fix: update prometheus port
manuelmauro Aug 13, 2024
5ac2a41
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 13, 2024
ac489db
docs: remove stale comment
manuelmauro Aug 13, 2024
fc93244
refactor: remove duplicated logs
manuelmauro Aug 13, 2024
edac83e
refactor: format code
manuelmauro Aug 13, 2024
fa672aa
refactor: extract external node run command to common crate
manuelmauro Aug 14, 2024
b6326f5
refactor: use common::external_node::run in zk_supervisor test all
manuelmauro Aug 14, 2024
fe53583
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 14, 2024
eefcd48
refactor: lint code
manuelmauro Aug 14, 2024
1a3c590
Fix healthcheck
Deniallugo Aug 14, 2024
1ef161b
Set to zero
Deniallugo Aug 14, 2024
81d55f4
refactor: try ss instead of netstat
manuelmauro Aug 14, 2024
a1d657a
fix: change dir before running ext node
manuelmauro Aug 14, 2024
8fcf801
refactor: remove duplicated code in zk_supervisor test build
manuelmauro Aug 14, 2024
33060f9
fix: use different wallets across different chains
manuelmauro Aug 15, 2024
1946708
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 15, 2024
2261684
fix: rely on process.env.MASTER_WALLET_PK in getMainWalletPk check
manuelmauro Aug 15, 2024
75c7007
fix: lint
manuelmauro Aug 15, 2024
671d6b2
fix: wrong config path for chain config
manuelmauro Aug 15, 2024
9931d43
fix: set MASTER_WALLET_PK
manuelmauro Aug 15, 2024
796e2b7
fix: add check for chain arg
manuelmauro Aug 15, 2024
9cb0058
feat: add `zk_supervisor test fund` subcommand
manuelmauro Aug 16, 2024
1d154e8
test: fund test wallets in CI
manuelmauro Aug 16, 2024
aacff8a
fix: wrong field naming in deserialization
manuelmauro Aug 19, 2024
22ff896
fix: serde rename unused fields
manuelmauro Aug 19, 2024
6624e26
refactor: better error management in subcommand
manuelmauro Aug 19, 2024
edb742a
feat: fund also test_mnemonic address
manuelmauro Aug 19, 2024
722a5c2
test: fund test wallets in reth config
manuelmauro Aug 19, 2024
9a7a877
refactor: fund test wallets within integration test
manuelmauro Aug 19, 2024
27d1414
refactor: set MASTER_WALLET_PK in zk_supervisor
manuelmauro Aug 19, 2024
e42ba41
refactor: lint
manuelmauro Aug 19, 2024
0599003
test: remove test wallets funding in reth config
manuelmauro Aug 19, 2024
9e1e02f
feat: remove `zk_supervisor test fund` subcommand
manuelmauro Aug 19, 2024
fc375aa
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 19, 2024
b650c11
feat: prefund test addresses in reth config
manuelmauro Aug 19, 2024
24c2c20
feat: check balance before funding test wallet
manuelmauro Aug 19, 2024
93871bf
refactor: fund test wallets using the default test wallet instead of …
manuelmauro Aug 19, 2024
fec3c95
fix: remove temporary log
manuelmauro Aug 19, 2024
0ba4e93
fix: remove temporary log
manuelmauro Aug 19, 2024
6f708cf
fix: wrong address encoding
manuelmauro Aug 19, 2024
db43e49
fix: correctly use test wallet
manuelmauro Aug 19, 2024
5decf6a
fix: compilation errors
manuelmauro Aug 19, 2024
07735d2
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 20, 2024
0110b3e
ci: run CI tests on multiple chains sequentially
manuelmauro Aug 20, 2024
2e76c5d
refactor: use MASTER_WALLET_PK env variable instead of sourcing walle…
manuelmauro Aug 20, 2024
7d9d25d
fix: keep previous SK derivation as fallback
manuelmauro Aug 20, 2024
20713e3
refactor: move TestWallets struct to utils mod
manuelmauro Aug 20, 2024
496e49f
fix: set MASTER_WALLET_PK whe running recovery tests
manuelmauro Aug 20, 2024
2726430
fix: hardcoded ports for external node
manuelmauro Aug 20, 2024
06c06b5
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 20, 2024
c327a6f
fix: lint
manuelmauro Aug 20, 2024
bb5ca1e
refactor: lint
manuelmauro Aug 20, 2024
11e0a13
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 20, 2024
e213b91
feat: add dapp-portal support to zk_inception (#2659)
sanekmelnikov Aug 20, 2024
aca5ef7
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
Deniallugo Aug 20, 2024
dd3a0a1
refactor: run validium tests before rollup ones
manuelmauro Aug 21, 2024
95c05c1
fix: run a single upgrade test ad the end of CI
manuelmauro Aug 21, 2024
dd84aa1
Update workflow
Deniallugo Aug 21, 2024
aa5b787
refactor: remove duplicated code
manuelmauro Aug 21, 2024
3cd9141
feat: generate test wallets incrementing base_path
manuelmauro Aug 21, 2024
450ea25
feat: add utility subcommand
manuelmauro Aug 21, 2024
52488b5
refactor: format code
manuelmauro Aug 21, 2024
e962052
refactor: lint
manuelmauro Aug 21, 2024
845ffeb
Some more adjustments
Deniallugo Aug 21, 2024
8c3c491
fix: comment out step in CI
manuelmauro Aug 21, 2024
21ade14
ci: redesigned CI pipeline for toolbox
manuelmauro Aug 21, 2024
e3bafdc
feat: run CI tests concurrently across chains
manuelmauro Aug 21, 2024
f636499
refactor: remove `zk_supervisor test all` subcommand
manuelmauro Aug 21, 2024
e722817
feat: prebuild dependencies in CI
manuelmauro Aug 21, 2024
8373f89
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 21, 2024
96b782c
fix: --no-deps option help message
manuelmauro Aug 21, 2024
c432fd2
Fix test for expensive calls
Deniallugo Aug 21, 2024
5428ef9
Fix contract submodule
Deniallugo Aug 21, 2024
afd6256
feat: redirect stdout and stderr from CI bg jobs to file
manuelmauro Aug 21, 2024
81df658
Fix db name
Deniallugo Aug 21, 2024
4b14ba0
Drastically exceed the timeout
Deniallugo Aug 21, 2024
825c54f
test: temporarely disable recovery tests
manuelmauro Aug 22, 2024
13458a4
feat: use MASTER_WALLET_PK in revert tests
manuelmauro Aug 22, 2024
e1fe9a6
refactor: lint
manuelmauro Aug 22, 2024
27e7178
fix: EN node URL
manuelmauro Aug 22, 2024
7a325b4
fix: external node URL
manuelmauro Aug 22, 2024
3af3557
refactor: format
manuelmauro Aug 22, 2024
2ea31ec
fix: wait for node responsiveness before deposit
manuelmauro Aug 22, 2024
30c48c7
ci: run revert tests sequentially
manuelmauro Aug 22, 2024
caa8b3e
fix: apply fix to revert test
manuelmauro Aug 22, 2024
2f0db11
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 23, 2024
9143997
fix: remove duplicated dependency
manuelmauro Aug 23, 2024
3b572a2
test: temporarily disable failing tests
manuelmauro Aug 23, 2024
92fd9b7
refactor: format
manuelmauro Aug 23, 2024
ad518d4
fix: ignore "server shutting down" error for a number of times
manuelmauro Aug 23, 2024
398038f
refactor: lint
manuelmauro Aug 23, 2024
c7776b3
fix: long running loop
manuelmauro Aug 23, 2024
659f5ae
refactor: format
manuelmauro Aug 23, 2024
d03b818
refactor: use era chain as rollup test chain
manuelmauro Aug 23, 2024
ef360c3
feat: apply default port offset based on chain ID
manuelmauro Aug 23, 2024
32da4b8
refactor: use newtype for port offset
manuelmauro Aug 23, 2024
a47effa
refactor: reduce code duplication
manuelmauro Aug 23, 2024
4d54fa6
fix(zk_toolbox): Set proper artifacts
Deniallugo Aug 22, 2024
7048155
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 26, 2024
2c3ebfc
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 27, 2024
ed54311
fix(zk_toolbox): Set proper artifacts (#2709)
Deniallugo Aug 27, 2024
57b1739
Downgrade the contracts
Deniallugo Aug 27, 2024
b55ee7c
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
Deniallugo Aug 27, 2024
fc505ae
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 28, 2024
c0c7cdf
feat(zk_toolbox): add custom token scenario to toolbox CI (#2731)
manuelmauro Aug 28, 2024
d73798c
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
Deniallugo Aug 28, 2024
f66bfd9
refactor: add vague return type
manuelmauro Aug 29, 2024
416cd16
refactor: wrap snapshot creation in function
manuelmauro Aug 29, 2024
2615187
feat: impl From<PortOffset> for u16
manuelmauro Aug 29, 2024
151b142
refactor: format
manuelmauro Aug 29, 2024
0708a8b
revert: remove code committed by mistake
manuelmauro Aug 29, 2024
5c6e385
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 29, 2024
98100cd
fix: await async function
manuelmauro Aug 29, 2024
b20a1fb
fix: refactor linter database url to use host network
manuelmauro Aug 29, 2024
ffe2ec1
Merge branch 'main' into manuel-add-multi-config-ci-integration-test
manuelmauro Aug 29, 2024
7879e2e
refactor: remove unused fields in TestWallets
manuelmauro Aug 29, 2024
7b3dfa7
fix: check for serverProcess initialization before killing
manuelmauro Aug 29, 2024
e7e4cfb
fix: if serverProcess is undefined default to killall
manuelmauro Aug 29, 2024
323206e
refactor: remove redundant log
manuelmauro Aug 29, 2024
7abd806
refactor: format code
manuelmauro Aug 29, 2024
f90064b
fix: make fileConfig a global variable
manuelmauro Aug 29, 2024
45fa3a0
fix: correctly ignore stdin in subprocesses
manuelmauro Aug 29, 2024
801a712
Return back fast runner
Deniallugo Aug 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci-core-lint-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ jobs:
echo ZKSYNC_HOME=$(pwd) >> $GITHUB_ENV
echo $(pwd)/bin >> $GITHUB_PATH
echo IN_DOCKER=1 >> .env
echo "prover_url=postgres://postgres:notsecurepassword@postgres:5432/zksync_local_prover" >> $GITHUB_ENV
echo "core_url=postgres://postgres:notsecurepassword@postgres:5432/zksync_local" >> $GITHUB_ENV
echo "prover_url=postgres://postgres:notsecurepassword@localhost:5432/zksync_local_prover" >> $GITHUB_ENV
echo "core_url=postgres://postgres:notsecurepassword@localhost:5432/zksync_local" >> $GITHUB_ENV

- name: Start services
run: |
Expand Down
27 changes: 14 additions & 13 deletions .github/workflows/ci-core-reusable.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
name: lint
uses: ./.github/workflows/ci-core-lint-reusable.yml
unit-tests:
runs-on: [matterlabs-ci-runner-highmem-long]
runs-on: [ matterlabs-ci-runner-highmem-long ]

steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
Expand Down Expand Up @@ -70,11 +70,11 @@ jobs:
ci_run zk f cargo test --release -p vm-benchmark --bench oneshot --bench batch

loadtest:
runs-on: [matterlabs-ci-runner-high-performance]
runs-on: [ matterlabs-ci-runner-high-performance ]
strategy:
fail-fast: false
matrix:
vm_mode: ["old", "new"]
vm_mode: [ "old", "new" ]

steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
Expand Down Expand Up @@ -142,13 +142,13 @@ jobs:
# To be consistent with the rest of the workflow we disable it explicitly.
fail-fast: false
matrix:
consensus: [false, true]
base_token: ["Eth", "Custom"]
deployment_mode: ["Rollup", "Validium"]
consensus: [ false, true ]
base_token: [ "Eth", "Custom" ]
deployment_mode: [ "Rollup", "Validium" ]
env:
SERVER_COMPONENTS: "api,tree,eth,state_keeper,housekeeper,commitment_generator,vm_runner_protective_reads,vm_runner_bwip,vm_playground,da_dispatcher${{ matrix.consensus && ',consensus' || '' }}${{ matrix.base_token == 'Custom' && ',base_token_ratio_persister' || '' }}"

runs-on: [matterlabs-ci-runner-highmem-long]
runs-on: [ matterlabs-ci-runner-highmem-long ]
steps:
- uses: actions/checkout@a5ac7e51b41094c92402da3b24376905380afc29 # v4
with:
Expand Down Expand Up @@ -243,15 +243,15 @@ jobs:
DEPLOYMENT_MODE=${{ matrix.deployment_mode }} \
SNAPSHOTS_CREATOR_VERSION=${{ matrix.deployment_mode == 'Validium' && '0' || '1' }} \
DISABLE_TREE_DURING_PRUNING=${{ matrix.base_token == 'Eth' }} \
ETH_CLIENT_WEB3_URL="http://reth:8545" \
ETH_CLIENT_WEB3_URL="http://localhost:8545" \
PASSED_ENV_VARS="ENABLE_CONSENSUS,DEPLOYMENT_MODE,DISABLE_TREE_DURING_PRUNING,SNAPSHOTS_CREATOR_VERSION,ETH_CLIENT_WEB3_URL" \
ci_run yarn recovery-test snapshot-recovery-test

- name: Genesis recovery test
run: |
ENABLE_CONSENSUS=${{ matrix.consensus }} \
DEPLOYMENT_MODE=${{ matrix.deployment_mode }} \
ETH_CLIENT_WEB3_URL="http://reth:8545" \
ETH_CLIENT_WEB3_URL="http://localhost:8545" \
PASSED_ENV_VARS="ENABLE_CONSENSUS,DEPLOYMENT_MODE,ETH_CLIENT_WEB3_URL" \
ci_run yarn recovery-test genesis-recovery-test

Expand Down Expand Up @@ -314,10 +314,10 @@ jobs:
strategy:
fail-fast: false
matrix:
consensus: [false, true]
base_token: ["Eth", "Custom"]
deployment_mode: ["Rollup", "Validium"]
runs-on: [matterlabs-ci-runner-highmem-long]
consensus: [ false, true ]
base_token: [ "Eth", "Custom" ]
deployment_mode: [ "Rollup", "Validium" ]
runs-on: [ matterlabs-ci-runner-highmem-long ]

env:
SERVER_COMPONENTS: "api,tree,eth,state_keeper,housekeeper,commitment_generator,vm_runner_protective_reads,vm_runner_bwip,da_dispatcher${{ matrix.consensus && ',consensus' || '' }}${{ matrix.base_token == 'Custom' && ',base_token_ratio_persister' || '' }}"
Expand Down Expand Up @@ -431,3 +431,4 @@ jobs:
run: |
ci_run sccache --show-stats || true
ci_run cat /tmp/sccache_log.txt || true

Loading
Loading