Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run bench on demo #1552

Merged
merged 115 commits into from
Aug 29, 2024
Merged
Changes from 1 commit
Commits
Show all changes
115 commits
Select commit Hold shift + click to select a range
fe73fd1
Define basic demo options
ffakenz Aug 6, 2024
86201c0
Extract genDatasetConstantUTxO
ffakenz Aug 6, 2024
ffaf84c
Extract bench scenario
ffakenz Aug 6, 2024
74ad553
Extract findRunningCardanoNode'
ffakenz Aug 6, 2024
8d1eb68
Add network id as part of the options
ffakenz Aug 6, 2024
85667fd
Draft base bench demo script
ffakenz Aug 6, 2024
22a0dbd
Fix seeding devnet on bench-demo given scripts were already published
ffakenz Aug 8, 2024
1a16408
Fix tx building for funding transaction
ffakenz Aug 8, 2024
03da04b
Add hydra signing keys as option args to bench-demo
ffakenz Aug 8, 2024
47f105c
Extend generator of constant utxo property
ffakenz Aug 12, 2024
6b5b52e
Trying to work out why large numbers of clients fail
noonio Aug 12, 2024
85c6f75
Limit the nbr of client keys on arbitrary for dataset
ffakenz Aug 12, 2024
f338f27
Return funds to faucet when it finishes
ffakenz Aug 13, 2024
8cb5813
Avoid passing network id to options and faucet keys on generators
ffakenz Aug 13, 2024
74d6896
Make bench-demo sim to run on top of previous run
ffakenz Aug 14, 2024
b6d53d8
Replace hydra-keys options by hydra-client hosts instead
ffakenz Aug 14, 2024
d1716b7
Start a github action for network testing
ch1bo Aug 14, 2024
a449194
Spin up things individually
ch1bo Aug 14, 2024
9e0041c
Acquire and upload logs
ch1bo Aug 14, 2024
1de6a0a
Make the hnode fallback in demo/seed-devnet.sh non-interactive tty
ch1bo Aug 14, 2024
0cc7f8a
Add invocation to produce traffic
ch1bo Aug 14, 2024
d1a7094
check socket permissions
noonio Aug 14, 2024
04f7724
ls not la
noonio Aug 14, 2024
8800ea3
tmate
noonio Aug 14, 2024
59d27da
tmate; socket file permissions
noonio Aug 14, 2024
6c8cc69
split setup and run benchmarks
noonio Aug 14, 2024
7039d7f
Run pumba and the benchmarks
noonio Aug 14, 2024
781200d
Line buffering; 100% loss on pumba
noonio Aug 14, 2024
7a5750e
tmate
noonio Aug 14, 2024
cca50cc
Rebase and make use new hydra-client options during CI
ffakenz Aug 14, 2024
a8c420f
Use docker version of pumba
noonio Aug 15, 2024
7d77a2f
Does it work with 20% loss?
noonio Aug 15, 2024
f79dbb4
See if it succeeds with 0% loss
noonio Aug 15, 2024
69ced33
What about 5%
noonio Aug 15, 2024
c1d15e7
Optional tmate'ing
noonio Aug 16, 2024
26998e8
Disable condition as it doesn't work
noonio Aug 16, 2024
4fc35a2
Try nix again; leave very long running time
noonio Aug 16, 2024
61b1e89
Try a sleep
noonio Aug 16, 2024
9703451
Horrible hack
noonio Aug 16, 2024
9e00319
Note about checking for pumba working
noonio Aug 16, 2024
336ae3f
Target node 1 for loss and limit target ips for node-2, and node-3
noonio Aug 16, 2024
9ad4b41
Try /24
noonio Aug 16, 2024
9530f1e
Multiple targets is right
noonio Aug 16, 2024
a0e18bc
Let's try 2%
noonio Aug 16, 2024
9c49594
Add now 5%
noonio Aug 16, 2024
b66a128
3%
noonio Aug 16, 2024
e6e0e2a
4%
noonio Aug 16, 2024
314ec58
Try to wait for alice peer disconnected from bob and carol nodes
ffakenz Aug 16, 2024
8dcb837
Executable; compatible with NixOS
noonio Aug 16, 2024
a9ff65b
Add persitency-dir options to hydra-node
ffakenz Aug 16, 2024
88e5bce
Use a special image built for netem; no more tcimage
noonio Aug 19, 2024
080d915
Add base network testing readme docs
ffakenz Aug 19, 2024
6290615
Use local functions to seed the network
ffakenz Aug 19, 2024
de10549
Do not seed neetwork before runnin bench-demo
ffakenz Aug 20, 2024
1c086e9
Remove FIXME as its not reproduceable
ffakenz Aug 20, 2024
58eab02
Minor refactor on generateOneTransfer
ffakenz Aug 20, 2024
088d661
Use random/self transfer generator functions
noonio Aug 20, 2024
e48b67a
Update demo-bech to take datasets from args
ffakenz Aug 20, 2024
4d1b2cb
Gen demo datasets
ffakenz Aug 20, 2024
eaef6ec
Refactor HydraClient to keep peer information
ffakenz Aug 21, 2024
b29df8f
dbg
ffakenz Aug 21, 2024
7be87ce
Try to use the pre-seeded client funds instead of faucet's
ffakenz Aug 21, 2024
6c517b7
Generate each client dataset from a different client funding tx
ffakenz Aug 21, 2024
5a61308
Try not to specify any output
ffakenz Aug 21, 2024
2a69816
Try using mkTxOutAutoBalance instead
ffakenz Aug 21, 2024
3a67f0c
Fix client utxo
ffakenz Aug 21, 2024
6eec715
Try not to specify any output ~ v2
ffakenz Aug 21, 2024
b8bfb3e
Go back to unbalanced outputs ~ v2
ffakenz Aug 21, 2024
99035a4
Make funding tx optional for datasets
ffakenz Aug 21, 2024
374147a
Try not to specify any output ~ v3
ffakenz Aug 21, 2024
0250340
Try using mkTxOutAutoBalance instead ~ v2
ffakenz Aug 21, 2024
5617254
WIP; Revert faucet; use buildRawTransaction
noonio Aug 21, 2024
ef399ad
Create workdir if it doesn't exist
noonio Aug 22, 2024
04648a9
WIP on generateDemoUTxO function; just missing witnesses.
noonio Aug 22, 2024
0d7c62c
Small refactors around types
noonio Aug 22, 2024
c17ecfb
WIP back to generating the dataset on the fly
noonio Aug 22, 2024
b98ad0a
Do not pre-seed devnet
ffakenz Aug 22, 2024
42d4cdf
Use other script for exporting pparams
noonio Aug 23, 2024
b761578
Short contestation period so that it closes fast
noonio Aug 23, 2024
79dfee8
add failure handling on tx processing while head is open
ffakenz Aug 23, 2024
0a15354
Compute pct of transactions confirmed
noonio Aug 23, 2024
e205443
4% should work
noonio Aug 23, 2024
b00a43c
Enhance workflow
ffakenz Aug 26, 2024
0872fa1
Some nice refactoring
ffakenz Aug 27, 2024
8e2c14d
Make watch for logs optional
ffakenz Aug 27, 2024
f77ef9b
Reframe wait for peer disconnected variable
noonio Aug 27, 2024
a855021
Experiment with a matrix
noonio Aug 27, 2024
faee8b8
Name in artifact
noonio Aug 27, 2024
0efbdc4
Expect-failure matrix configuration
noonio Aug 27, 2024
7335112
Add failure mode we expect; refine matrix
noonio Aug 27, 2024
c80281f
Fix order of detecting error conditions
noonio Aug 27, 2024
a1e0de4
Add scaling factor to matrix config
noonio Aug 27, 2024
d14576b
Replace yq by docker inspect
ffakenz Aug 27, 2024
b90ee87
Upload the results as part of the artifacts
ffakenz Aug 27, 2024
46746c6
Make sure results.csv is written to the outputDirectory not the tmp d…
ffakenz Aug 27, 2024
8324d53
Write the summary out even when it failed
ffakenz Aug 27, 2024
0ccb72d
Make peers part of the matrix
noonio Aug 28, 2024
dd2051f
Allow 5% to fail
noonio Aug 28, 2024
b4c3376
Remove explicit failure tracking in the includes/excludes
noonio Aug 28, 2024
d7b18ab
Add extra scaling factor
noonio Aug 28, 2024
fc26227
Matrix information in the name
noonio Aug 28, 2024
4f31cfe
Rename hydra clients peers to api hosts
ffakenz Aug 28, 2024
cc59644
Update README
ffakenz Aug 28, 2024
116976f
Enhance user error during processTransactions
ffakenz Aug 28, 2024
d1d96fd
Update bench demo option description
ffakenz Aug 28, 2024
a5185a5
Make the scenario to seed the network
ffakenz Aug 28, 2024
02d460d
Rever changes over the benchmarks slug used by the website
ffakenz Aug 28, 2024
36ac132
Do not trace signing keys during benchmark
ffakenz Aug 28, 2024
cdf5ff6
Remove watch logs step and simplify workflow args
ffakenz Aug 28, 2024
5e69dd7
Remove unnecessary scripts
ffakenz Aug 28, 2024
a555aec
Enhance failure message for HUnitFailure errors
ffakenz Aug 28, 2024
3aafd9c
Remove hack to check every node sees head is initialized for demo
ffakenz Aug 29, 2024
710d65f
Refactor hacky boolean blindness to manage query params on withConnec…
ffakenz Aug 29, 2024
6654d08
Fix not seeding during scenario but before it gets executed
ffakenz Aug 29, 2024
c5d1208
Don't call --random on pumba; note about time
noonio Aug 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add scaling factor to matrix config
noonio authored and ch1bo committed Aug 28, 2024
commit a1e0de463d3e91d4f0363073f0f8c3c31a1663b6
22 changes: 11 additions & 11 deletions .github/workflows/network-test.yaml
Original file line number Diff line number Diff line change
@@ -9,11 +9,6 @@ on:
description: 'Run the build with tmate debugging enabled (https://github.com/marketplace/actions/debugging-with-tmate)'
required: false
default: false
scaling_factor:
type: number
description: 'Scaling factor for the benchmarks'
required: false
default: 10
wait_peer_disconnected:
type: boolean
description: 'Check other peers see the target peer is disconnected before running benchmark.'
@@ -34,19 +29,24 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
scaling_factor: [10]
netem_loss: [0, 1, 2, 3, 4, 5, 10, 20]
expect_failure: [false]
# We know these will fail; so don't try.
exclude:
- netem_loss: 10
- scaling_factor: 10
netem_loss: 10
expect_failure: false
- netem_loss: 20
- scaling_factor: 10
netem_loss: 20
expect_failure: false
# In fact, allow them to fail.
include:
- netem_loss: 10
- scaling_factor: 10
netem_loss: 10
expect_failure: true
- netem_loss: 20
- scaling_factor: 10
netem_loss: 20
expect_failure: true
steps:
- uses: actions/checkout@v4
@@ -109,7 +109,7 @@ jobs:
run: |
# Extract inputs with defaults for non-workflow_dispatch events
percent="${{ matrix.netem_loss }}"
scaling_factor="${{ github.event.inputs.scaling_factor || '10' }}"
scaling_factor="${{ matrix.scaling_factor }}"
target_peer="${{ github.event.inputs.target_peer || 'alice' }}"

peers_info_json=$(
@@ -146,6 +146,6 @@ jobs:
if: always()
uses: actions/upload-artifact@v4
with:
name: "docker-logs-netem-loss=${{ matrix.netem_loss }}"
name: "docker-logs-netem-loss=${{ matrix.netem_loss }},scaling_factor=${{ matrix.scaling_factor }}"
path: demo/docker-logs
if-no-files-found: ignore