Display difference in tx-cost outputs as a comment on PRs

[noonio]

This adds a simple diff calculation to the PR comment, just like the present "Transaction costs" comment we get, but instead a diff between the PR and the master branch.

Positive numbers are bad - it means the cost/size/etc got larger with the introduction of the PR. These are coloured black.

Negative numbers are good - it means a reduction in size/cost/etc, so these are coloured green.

Values that are the same between PR and branch are simply listed as -; i.e. unchanged.

Note

It seems that some differences are expected even when nothing relevant has changed, as in this PR itself. So some care will need to be taken in learning how to interpret these numbers, as we see them develop over furture PRs.

[github-actions]

Transaction costs

Sizes and execution budgets for Hydra protocol transactions. Note that unlisted parameters are currently using arbitrary values and results are not fully deterministic and comparable to previous runs.

Metadata
Generated at	2024-10-14 06:11:14.14569368 UTC
Max. memory units	14000000
Max. CPU units	10000000000
Max. tx size (kB)	16384

Script summary

Name	Hash	Size (Bytes)
νInitial	b512161ccb0652d7e9a0b540e4a3c808f73d6558a4bcabf374d85880	3969
νCommit	ea444d37d226e71eef73ac78d149750da977feb588900135bf9e8221	692
νHead	2253ddd95837c7aacc8635a971caaea743434152dd8dd2849bdf4162	10797
μHead	4d648ca239040b0e87901835aa11423e7aa3bd947ce6befe7db1bae8*	4508
νDeposit	1a011f23b139a6426767026bde10319546485d553219a5848cdac4e5	2993

• The minting policy hash is only usable for comparison. As the script is parameterized, the actual script is unique per head.

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	5094	5.61	2.21	0.44
2	5295	7.13	2.82	0.46
3	5500	8.55	3.38	0.49
5	5902	11.32	4.48	0.53
10	6908	18.10	7.16	0.65
57	16353	83.00	32.84	1.78

Commit transaction costs

This uses ada-only outputs for better comparability.

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	569	10.84	4.26	0.29
2	758	14.31	5.80	0.34
3	947	17.92	7.39	0.39
5	1322	25.56	10.73	0.49
10	2257	47.11	19.97	0.77
19	3938	94.71	39.81	1.38

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	57	560	19.87	7.59	0.39
2	113	671	28.85	10.97	0.49
3	170	782	36.31	13.81	0.58
4	226	893	47.09	17.87	0.70
5	282	1004	53.50	20.33	0.77
6	338	1116	61.35	23.32	0.86
7	393	1227	77.06	29.18	1.04
8	452	1342	72.99	27.80	1.00
9	506	1449	78.52	29.96	1.06
10	560	1564	86.68	33.07	1.16

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	657	18.68	8.18	0.39
2	757	19.35	9.15	0.41
3	968	21.52	10.75	0.44
5	1240	24.37	13.42	0.50
10	2010	33.33	20.76	0.66
48	7742	97.22	74.94	1.82

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	644	20.83	9.31	0.41
2	809	22.63	11.06	0.45
3	975	24.11	12.53	0.48
5	1320	27.88	16.09	0.55
10	1977	35.12	23.28	0.69
50	8133	99.89	86.55	1.94

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	679	26.81	11.48	0.48
2	842	28.53	13.04	0.51
3	943	30.35	14.59	0.54
5	1356	34.85	18.54	0.63
10	2012	43.17	26.03	0.78
39	6499	99.84	75.77	1.79

Abort transaction costs

There is some variation due to the random mixture of initial and already committed outputs.

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	4985	15.43	6.61	0.54
2	5111	21.73	9.35	0.62
3	5207	28.53	12.30	0.70
4	5455	36.29	15.85	0.80
5	5463	39.85	17.18	0.84
6	5574	45.78	19.78	0.91
7	5681	55.42	24.00	1.02
8	6112	65.81	28.92	1.16
9	6178	71.89	31.54	1.23
10	6178	74.93	32.68	1.26
11	6438	86.07	37.71	1.40
12	6474	89.13	38.84	1.44
13	6625	97.52	42.53	1.54
15	6498	99.71	42.70	1.55

FanOut transaction costs

Involves spending head output and burning head tokens. Uses ada-only UTxO for better comparability.

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	0	0	5089	10.19	4.26	0.49
10	1	57	5123	11.16	4.90	0.50
10	5	284	5259	15.83	7.79	0.57
10	10	568	5428	21.84	11.48	0.65
10	20	1139	5768	33.16	18.55	0.81
10	30	1710	6111	45.21	25.95	0.97
10	40	2277	6448	57.02	33.24	1.13
10	50	2848	6789	68.63	40.45	1.30
10	76	4324	7667	98.68	59.14	1.71

End-to-end benchmark results

This page is intended to collect the latest end-to-end benchmark results produced by Hydra's continuous integration (CI) system from the latest master code.

Please note that these results are approximate as they are currently produced from limited cloud VMs and not controlled hardware. Rather than focusing on the absolute results, the emphasis should be on relative results, such as how the timings for a scenario evolve as the code changes.

Generated at 2024-10-14 06:12:54.34733395 UTC

Baseline Scenario

Number of nodes	1
Number of txs	300
Avg. Confirmation Time (ms)	5.222795416
P99	10.082973439999995ms
P95	6.651513650000001ms
P50	4.9907825ms
Number of Invalid txs	0

Three local nodes

Number of nodes	3
Number of txs	900
Avg. Confirmation Time (ms)	22.766152485
P99	116.95122445999998ms
P95	30.358121399999998ms
P50	20.231026ms
Number of Invalid txs	0

[github-actions]

Transaction cost differences

Script summary

Name	Size (Bytes)
νInitial	-
νCommit	-
νHead	-
μHead	-
νDeposit	-

Init transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	+7	$${\color{green}-0.06}$$	$${\color{green}-0.02}$$	-
2	$${\color{green}-1.00}$$	$${\color{green}-0.03}$$	$${\color{green}-0.01}$$	-
3	+7	$${\color{green}-0.09}$$	$${\color{green}-0.04}$$	-
5	$${\color{green}-1.00}$$	+0.19	+0.09	-
10	+3	-	-	-
57	-	$${\color{green}-0.10}$$	$${\color{green}-0.04}$$	-

Commit transaction costs

UTxO	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	-	-	-
2	-	-	-	-
3	+3	-	-	-
5	+2	-	-	-
10	+7	-	-	-
19	-	-	-	-

CollectCom transaction costs

Parties	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	$${\color{green}-4.00}$$	-	-	-
2	$${\color{green}-1.00}$$	-	$${\color{green}-0.91}$$	$${\color{green}-0.33}$$	$${\color{green}-0.01}$$
3	-	-	+3.94	+1.46	+0.04
4	-	-	+5.45	+2.0	+0.06
5	+1.0	-	$${\color{green}-1.89}$$	$${\color{green}-0.71}$$	$${\color{green}-0.02}$$
6	+1.0	-	+8.97	+3.29	+0.09
7	+3.0	-	$${\color{green}-10.58}$$	$${\color{green}-3.92}$$	$${\color{green}-0.12}$$
8	$${\color{green}-2.00}$$	-	+8.11	+2.94	+0.09
9	+1.0	-	+9.52	+3.58	+0.1

Cost of Decrement Transaction

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	$${\color{green}-19.00}$$	+0.28	+0.09	-
2	+71	+1.09	+0.41	+0.02
3	$${\color{green}-46.00}$$	$${\color{green}-0.68}$$	$${\color{green}-0.25}$$	$${\color{green}-0.01}$$
5	$${\color{green}-127.00}$$	$${\color{green}-2.48}$$	$${\color{green}-1.00}$$	$${\color{green}-0.04}$$
10	+93	$${\color{green}-0.85}$$	$${\color{green}-0.39}$$	-
50	$${\color{green}-171.00}$$	$${\color{green}-2.92}$$	$${\color{green}-1.11}$$	$${\color{green}-0.04}$$

Close transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	-	$${\color{green}-0.04}$$	$${\color{green}-0.02}$$	-
2	+46	+0.37	+0.32	+0.01
3	$${\color{green}-4.00}$$	-	-	-
5	$${\color{green}-74.00}$$	$${\color{green}-0.41}$$	$${\color{green}-0.37}$$	-
10	$${\color{green}-44.00}$$	+0.15	+0.02	-
50	+274	+1.56	+1.68	+0.04

Contest transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	+29.0	+0.05	+0.04	-
2	$${\color{green}-45.00}$$	$${\color{green}-0.45}$$	$${\color{green}-0.35}$$	$${\color{green}-0.01}$$
3	$${\color{green}-90.00}$$	$${\color{green}-0.32}$$	$${\color{green}-0.34}$$	$${\color{green}-0.01}$$
5	$${\color{green}-133.00}$$	$${\color{green}-1.11}$$	$${\color{green}-0.97}$$	$${\color{green}-0.03}$$
10	+333.0	+1.75	+1.67	+0.05

Abort transaction costs

Parties	Tx size	% max Mem	% max CPU	Min fee ₳
1	+81	+2.86	+1.47	+0.04
2	+41	+2.62	+1.26	+0.03
3	$${\color{green}-22.00}$$	$${\color{green}-1.85}$$	$${\color{green}-0.87}$$	$${\color{green}-0.02}$$
4	$${\color{green}-132.00}$$	$${\color{green}-4.31}$$	$${\color{green}-2.13}$$	$${\color{green}-0.06}$$
5	$${\color{green}-125.00}$$	$${\color{green}-6.16}$$	$${\color{green}-3.01}$$	$${\color{green}-0.08}$$
6	+209	+5.06	+2.56	+0.07
7	+108	+6.48	+3.13	+0.08
8	$${\color{green}-191.00}$$	$${\color{green}-0.93}$$	$${\color{green}-0.62}$$	$${\color{green}-0.02}$$
9	+82	+0.54	+0.39	+0.01
10	$${\color{green}-201.00}$$	$${\color{green}-8.18}$$	$${\color{green}-3.93}$$	$${\color{green}-0.10}$$
11	+230	$${\color{green}-0.95}$$	$${\color{green}-0.16}$$	-
12	$${\color{green}-26.00}$$	+4.63	+1.9	+0.05

FanOut transaction costs

Parties	UTxO	UTxO (bytes)	Tx size	% max Mem	% max CPU	Min fee ₳
10	-	-	-	$${\color{green}-0.59}$$	$${\color{green}-0.25}$$	$${\color{green}-0.01}$$
10	-	-	$${\color{green}-1.00}$$	+0.39	+0.17	-
10	-	-	-	+1.08	+0.46	+0.01
10	-	$${\color{green}-2.00}$$	$${\color{green}-2.00}$$	+0.2	+0.08	-
10	-	-	-	+0.67	+0.29	-
10	-	+5	+5	+0.2	+0.09	-
10	-	$${\color{green}-3.00}$$	$${\color{green}-3.00}$$	+0.2	+0.09	+0.01
10	-	+3	+3	+0.08	+0.04	-
10	+1	+55	+32	+0.49	+0.43	+0.01

[github-actions]

Test Results

544 tests  ±0   538 ✅ ±0   26m 30s ⏱️ +45s
162 suites ±0     6 💤 ±0 
  7 files   ±0     0 ❌ ±0 

Results for commit 90f61d4. ± Comparison against base commit d8c0a49.

♻️ This comment has been updated with latest results.