self-hosted CI

[locallycompact]

This replaces our CI system from ephemeral github runners to bare metal NixOS machines.

This has the advantage of a persistent nix store, so runners do not need to synchronise dependencies and shell environments.

In order to stop processes competing for ports, the hydra-node and hydra-cluster tests have been converted to nixos vm tests. This requires qemu on the host, and so cloud runners would struggle to run it.

However, in the event that the main runner goes down - there is a system available for deployment at https://github.com/cardano-scaling/hydra-github-runner

This also introduces cachix-push instead of cachix github action, which is run on master after everything has built.

[github-actions]

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-09-19 13:09:55.18664742 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	2fac819a1f4f14e29639d1414220d2a18b6abd6b8e444d88d0dda8ff	3799
νCommit	2043a9f1a685bcf491413a5f139ee42e335157c8c6bc8d9e4018669d	1743
νHead	2ee477c60839936be49a50030690865b5bed4db8cd2f05bf255ac680	10068
μHead	a1610f6e64843161f4a88229c0286176f5325de3e2f773eec2b1d818*	4508
νDeposit	c2117fd9ebdee3e96b81fd67ff7092d638926415c10f1f7e5a267ad0	2791

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5096	5.61	2.21	0.44
2	5297	7.40	2.93	0.46
3	5502	8.37	3.30	0.48
5	5901	11.17	4.41	0.53
10	6907	18.02	7.12	0.65
57	16355	82.91	32.79	1.78

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	569	10.52	4.15	0.29
2	759	13.86	5.65	0.34
3	943	17.33	7.20	0.38
5	1317	24.65	10.44	0.48
10	2246	45.22	19.36	0.75
20	4132	95.99	40.76	1.40

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	560	21.46	8.41	0.41
2	114	671	32.95	13.05	0.54
3	171	782	46.25	18.49	0.69
4	226	897	59.03	23.85	0.84
5	282	1004	79.59	32.21	1.07
6	338	1116	93.61	38.27	1.23

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	651	17.95	7.89	0.38
2	728	17.82	8.52	0.39
3	862	19.17	9.78	0.41
5	1230	23.56	12.92	0.49
10	1960	30.96	19.45	0.63
48	7566	93.37	71.46	1.75

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	649	20.07	9.01	0.41
2	781	21.53	10.41	0.43
3	976	23.37	12.15	0.47
5	1136	25.46	14.26	0.51
10	2085	35.08	23.23	0.69
49	8108	99.26	83.77	1.91

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	684	25.89	11.14	0.47
2	876	28.03	12.97	0.51
3	1031	29.91	14.55	0.54
5	1268	33.24	17.46	0.60
10	2007	42.69	25.46	0.77
39	6297	97.46	72.39	1.74

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4983	17.43	7.59	0.57
2	5094	28.31	12.41	0.69
3	5331	43.56	19.36	0.88
4	5482	59.47	26.46	1.06
5	5351	60.23	26.34	1.06
6	5471	83.59	36.89	1.33

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
5	0	0	4934	7.70	3.25	0.45
5	1	57	4968	8.72	3.92	0.47
5	5	284	5104	12.96	6.64	0.53
5	10	570	5275	19.29	10.49	0.61
5	20	1139	5613	30.13	17.40	0.77
5	30	1708	5954	41.65	24.61	0.93
5	40	2280	6297	52.98	31.73	1.08
5	50	2846	6633	64.51	38.94	1.24
5	81	4614	7687	99.47	60.99	1.73

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-09-19 13:12:00.069125689 UTC

Baseline Scenario

Number of nodes	1
Number of txs	3000
Avg. Confirmation Time (ms)	10.699220592
P99	15.188380989999999ms
P95	13.208096599999998ms
P50	10.519141000000001ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	9000
Avg. Confirmation Time (ms)	24.463144047
P99	31.850371730000006ms
P95	28.869142850000003ms
P50	24.170907ms
Number of Invalid txs	0

[github-actions]

Test Results

49 tests   - 454   49 ✅  - 448   2m 48s ⏱️ - 19m 33s
23 suites  - 137    0 💤  -   6 
 4 files    -   3    0 ❌ ±  0 

Results for commit c30356d. ± Comparison against base commit f72a220.

This pull request removes 454 tests.

Hydra.API.ClientInput/FromJSON (ValidatedTx era) ‑ accepts transactions produced via cardano-cli
Hydra.API.ClientInput/JSON encoding of (ReasonablySized (ClientInput (Tx ConwayEra))) ‑ allows to encode values with aeson and read them back
Hydra.API.ClientInput/JSON encoding of (ReasonablySized (ClientInput (Tx ConwayEra))) ‑ produces the same JSON as is found in golden/ReasonablySized (ClientInput (Tx ConwayEra)).json
Hydra.API.ClientInput/JSON encoding of (ReasonablySized (ClientInput SimpleTx)) ‑ allows to encode values with aeson and read them back
Hydra.API.ClientInput/JSON encoding of (ReasonablySized (ClientInput SimpleTx)) ‑ produces the same JSON as is found in golden/ReasonablySized (ClientInput SimpleTx).json
Hydra.API.HTTPServer ‑ Validate /cardano-transaction publish api schema
Hydra.API.HTTPServer ‑ Validate /cardano-transaction subscribe api schema
Hydra.API.HTTPServer ‑ Validate /commit publish api schema
Hydra.API.HTTPServer ‑ Validate /commit subscribe api schema
Hydra.API.HTTPServer ‑ Validate /decommit publish api schema
…

♻️ This comment has been updated with latest results.