Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: timeout sv-1 test #1099

Merged
merged 1 commit into from
Sep 4, 2024
Merged

Conversation

average-gary
Copy link
Contributor

Adds a basic 10 minute timeout to the sv-1 test. More investigation would be worth doing.
#1094

pavlenex

This comment was marked as off-topic.

Copy link
Contributor

github-actions bot commented Aug 14, 2024

🐰Bencher

ReportTue, September 3, 2024 at 17:06:33 UTC
ProjectStratum V2 (SRI)
Branchsv1-test-limiter
Testbedsv2

🚨 3 ALERTS: Threshold Boundary Limits exceeded!
BenchmarkMeasure (units)ViewValueLower BoundaryUpper Boundary
client_sv2_mining_message_submit_standardLatency (nanoseconds (ns))🚨 (view plot | view alert)14.80 (+0.94%)14.58 (98.54%)14.74 (100.40%)
client_sv2_mining_message_submit_standard_serializeLatency (nanoseconds (ns))🚨 (view plot | view alert)308.58 (+15.47%)244.63 (79.28%)289.82 (106.47%)
client_sv2_mining_message_submit_standard_serialize_deserializeLatency (nanoseconds (ns))🚨 (view plot | view alert)638.18 (+7.37%)555.20 (87.00%)633.58 (100.73%)

Click to view all benchmark results
BenchmarkLatencyLatency Results
nanoseconds (ns) | (Δ%)
Latency Lower Boundary
nanoseconds (ns) | (%)
Latency Upper Boundary
nanoseconds (ns) | (%)
client_sv2_handle_message_common✅ (view plot)44.60 (-0.09%)43.52 (97.58%)45.76 (97.47%)
client_sv2_handle_message_mining✅ (view plot)74.53 (+0.79%)64.13 (86.04%)83.78 (88.97%)
client_sv2_mining_message_submit_standard🚨 (view plot | view alert)14.80 (+0.94%)14.58 (98.54%)14.74 (100.40%)
client_sv2_mining_message_submit_standard_serialize🚨 (view plot | view alert)308.58 (+15.47%)244.63 (79.28%)289.82 (106.47%)
client_sv2_mining_message_submit_standard_serialize_deserialize🚨 (view plot | view alert)638.18 (+7.37%)555.20 (87.00%)633.58 (100.73%)
client_sv2_open_channel✅ (view plot)169.09 (+1.87%)159.19 (94.15%)172.79 (97.86%)
client_sv2_open_channel_serialize✅ (view plot)271.48 (-3.45%)265.26 (97.71%)297.10 (91.38%)
client_sv2_open_channel_serialize_deserialize✅ (view plot)379.82 (+0.58%)324.46 (85.42%)430.81 (88.16%)
client_sv2_setup_connection✅ (view plot)169.40 (+3.63%)150.16 (88.64%)176.76 (95.83%)
client_sv2_setup_connection_serialize✅ (view plot)455.83 (-2.95%)412.47 (90.49%)526.87 (86.52%)
client_sv2_setup_connection_serialize_deserialize✅ (view plot)1,032.00 (+4.85%)883.49 (85.61%)1,085.06 (95.11%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

Copy link
Contributor

github-actions bot commented Aug 14, 2024

🐰Bencher

ReportTue, September 3, 2024 at 17:06:36 UTC
ProjectStratum V2 (SRI)
Branchsv1-test-limiter
Testbedsv2
Click to view all benchmark results
BenchmarkEstimated CyclesEstimated Cycles Results
estimated cycles | (Δ%)
Estimated Cycles Lower Boundary
estimated cycles | (%)
Estimated Cycles Upper Boundary
estimated cycles | (%)
InstructionsInstructions Results
instructions | (Δ%)
Instructions Lower Boundary
instructions | (%)
Instructions Upper Boundary
instructions | (%)
L1 AccessesL1 Accesses Results
accesses | (Δ%)
L1 Accesses Lower Boundary
accesses | (%)
L1 Accesses Upper Boundary
accesses | (%)
L2 AccessesL2 Accesses Results
accesses | (Δ%)
L2 Accesses Lower Boundary
accesses | (%)
L2 Accesses Upper Boundary
accesses | (%)
RAM AccessesRAM Accesses Results
accesses | (Δ%)
RAM Accesses Lower Boundary
accesses | (%)
RAM Accesses Upper Boundary
accesses | (%)
client_sv2_handle_message_common✅ (view plot)2,125.00 (+2.72%)1,974.16 (92.90%)2,163.35 (98.23%)✅ (view plot)473.00 (+0.33%)457.42 (96.71%)485.47 (97.43%)✅ (view plot)735.00 (+0.24%)712.02 (96.87%)754.49 (97.42%)✅ (view plot)5.00 (-19.25%)-0.55 (-10.90%)12.93 (38.67%)✅ (view plot)39.00 (+4.63%)34.65 (88.85%)39.89 (97.76%)
client_sv2_handle_message_mining✅ (view plot)8,146.00 (-0.68%)8,077.02 (99.15%)8,325.95 (97.84%)✅ (view plot)2,137.00 (+0.22%)2,093.23 (97.95%)2,171.59 (98.41%)✅ (view plot)3,166.00 (+0.42%)3,089.52 (97.58%)3,216.17 (98.44%)✅ (view plot)30.00 (-18.32%)29.63 (98.77%)43.82 (68.45%)✅ (view plot)138.00 (-0.72%)136.21 (98.70%)141.79 (97.33%)
client_sv2_mining_message_submit_standard✅ (view plot)6,277.00 (-0.04%)6,169.90 (98.29%)6,388.77 (98.25%)✅ (view plot)1,750.00 (-0.01%)1,736.86 (99.25%)1,763.46 (99.24%)✅ (view plot)2,552.00 (-0.06%)2,532.50 (99.24%)2,574.55 (99.12%)✅ (view plot)17.00 (-1.35%)11.16 (65.62%)23.31 (72.93%)✅ (view plot)104.00 (+0.01%)101.01 (97.13%)106.97 (97.23%)
client_sv2_mining_message_submit_standard_serialize✅ (view plot)14,700.00 (-0.43%)14,524.28 (98.80%)15,002.25 (97.99%)✅ (view plot)4,694.00 (-0.00%)4,680.86 (99.72%)4,707.46 (99.71%)✅ (view plot)6,760.00 (+0.05%)6,736.39 (99.65%)6,776.50 (99.76%)✅ (view plot)41.00 (-8.81%)36.61 (89.29%)53.31 (76.91%)✅ (view plot)221.00 (-0.60%)215.64 (97.57%)229.05 (96.49%)
client_sv2_mining_message_submit_standard_serialize_deserialize✅ (view plot)27,554.00 (+0.17%)27,183.85 (98.66%)27,829.12 (99.01%)✅ (view plot)10,585.00 (+0.22%)10,506.58 (99.26%)10,616.07 (99.71%)✅ (view plot)15,404.00 (+0.23%)15,288.30 (99.25%)15,447.50 (99.72%)✅ (view plot)78.00 (-4.81%)74.90 (96.02%)88.98 (87.66%)✅ (view plot)336.00 (+0.27%)326.10 (97.05%)344.12 (97.64%)
client_sv2_open_channel✅ (view plot)4,405.00 (-0.76%)4,252.33 (96.53%)4,624.82 (95.25%)✅ (view plot)1,461.00 (+0.05%)1,447.46 (99.07%)1,473.05 (99.18%)✅ (view plot)2,160.00 (+0.21%)2,135.41 (98.86%)2,175.58 (99.28%)✅ (view plot)8.00 (-24.50%)4.74 (59.20%)16.46 (48.61%)✅ (view plot)63.00 (-1.13%)58.72 (93.21%)68.71 (91.68%)
client_sv2_open_channel_serialize✅ (view plot)13,992.00 (-1.06%)13,840.88 (98.92%)14,444.33 (96.87%)✅ (view plot)5,064.00 (+0.01%)5,050.46 (99.73%)5,076.05 (99.76%)✅ (view plot)7,332.00 (+0.14%)7,299.39 (99.56%)7,343.90 (99.84%)✅ (view plot)30.00 (-15.07%)28.71 (95.71%)41.93 (71.54%)✅ (view plot)186.00 (-2.02%)181.43 (97.54%)198.25 (93.82%)
client_sv2_open_channel_serialize_deserialize✅ (view plot)22,657.00 (+0.06%)22,316.09 (98.50%)22,970.62 (98.63%)✅ (view plot)8,027.00 (+0.31%)7,949.13 (99.03%)8,055.70 (99.64%)✅ (view plot)11,677.00 (+0.30%)11,562.98 (99.02%)11,720.29 (99.63%)✅ (view plot)75.00 (+1.98%)64.28 (85.71%)82.81 (90.57%)✅ (view plot)303.00 (-0.27%)294.61 (97.23%)313.05 (96.79%)
client_sv2_setup_connection✅ (view plot)4,703.00 (+0.31%)4,614.77 (98.12%)4,762.38 (98.75%)✅ (view plot)1,502.00 (+0.05%)1,488.46 (99.10%)1,514.05 (99.20%)✅ (view plot)2,278.00 (+0.05%)2,256.45 (99.05%)2,297.27 (99.16%)✅ (view plot)9.00 (-4.09%)4.09 (45.48%)14.67 (61.33%)✅ (view plot)68.00 (+0.64%)65.46 (96.27%)69.67 (97.60%)
client_sv2_setup_connection_serialize✅ (view plot)16,170.00 (-0.28%)15,971.31 (98.77%)16,460.73 (98.23%)✅ (view plot)5,963.00 (+0.01%)5,949.46 (99.77%)5,975.05 (99.80%)✅ (view plot)8,665.00 (+0.07%)8,635.07 (99.65%)8,682.43 (99.80%)✅ (view plot)38.00 (-9.87%)31.67 (83.35%)52.65 (72.17%)✅ (view plot)209.00 (-0.43%)203.37 (97.31%)216.43 (96.57%)
client_sv2_setup_connection_serialize_deserialize✅ (view plot)35,616.00 (+0.20%)35,356.15 (99.27%)35,734.36 (99.67%)✅ (view plot)14,855.00 (+0.17%)14,775.45 (99.46%)14,884.26 (99.80%)✅ (view plot)21,821.00 (+0.18%)21,692.49 (99.41%)21,873.06 (99.76%)✅ (view plot)92.00 (-3.20%)77.34 (84.06%)112.74 (81.60%)✅ (view plot)381.00 (+0.36%)375.25 (98.49%)384.03 (99.21%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

Copy link
Contributor

github-actions bot commented Aug 14, 2024

🐰Bencher

ReportTue, September 3, 2024 at 17:06:37 UTC
ProjectStratum V2 (SRI)
Branchsv1-test-limiter
Testbedsv1
Click to view all benchmark results
BenchmarkEstimated CyclesEstimated Cycles Results
estimated cycles | (Δ%)
Estimated Cycles Lower Boundary
estimated cycles | (%)
Estimated Cycles Upper Boundary
estimated cycles | (%)
InstructionsInstructions Results
instructions | (Δ%)
Instructions Lower Boundary
instructions | (%)
Instructions Upper Boundary
instructions | (%)
L1 AccessesL1 Accesses Results
accesses | (Δ%)
L1 Accesses Lower Boundary
accesses | (%)
L1 Accesses Upper Boundary
accesses | (%)
L2 AccessesL2 Accesses Results
accesses | (Δ%)
L2 Accesses Lower Boundary
accesses | (%)
L2 Accesses Upper Boundary
accesses | (%)
RAM AccessesRAM Accesses Results
accesses | (Δ%)
RAM Accesses Lower Boundary
accesses | (%)
RAM Accesses Upper Boundary
accesses | (%)
get_authorize✅ (view plot)8,545.00 (+1.25%)8,158.06 (95.47%)8,720.71 (97.99%)✅ (view plot)3,772.00 (+0.67%)3,636.61 (96.41%)3,856.82 (97.80%)✅ (view plot)5,295.00 (+0.76%)5,103.97 (96.39%)5,405.68 (97.95%)✅ (view plot)6.00 (-18.79%)4.33 (72.17%)10.45 (57.43%)✅ (view plot)92.00 (+2.30%)85.88 (93.35%)93.98 (97.89%)
get_submit✅ (view plot)95,442.00 (-0.06%)94,918.93 (99.45%)96,086.20 (99.33%)✅ (view plot)59,522.00 (+0.09%)59,200.81 (99.46%)59,739.54 (99.64%)✅ (view plot)85,507.00 (+0.11%)85,034.96 (99.45%)85,791.43 (99.67%)✅ (view plot)48.00 (-3.46%)33.16 (69.09%)66.27 (72.43%)✅ (view plot)277.00 (-1.48%)273.95 (98.90%)288.38 (96.05%)
get_subscribe✅ (view plot)8,064.00 (+0.79%)7,713.25 (95.65%)8,289.04 (97.29%)✅ (view plot)2,848.00 (+0.46%)2,729.34 (95.83%)2,940.37 (96.86%)✅ (view plot)3,984.00 (+0.54%)3,821.56 (95.92%)4,103.45 (97.09%)✅ (view plot)11.00 (-19.53%)6.20 (56.37%)21.14 (52.04%)✅ (view plot)115.00 (+1.38%)108.75 (94.56%)118.13 (97.35%)
serialize_authorize✅ (view plot)12,328.00 (+0.70%)11,935.35 (96.81%)12,550.32 (98.23%)✅ (view plot)5,343.00 (+0.48%)5,207.61 (97.47%)5,427.82 (98.44%)✅ (view plot)7,458.00 (+0.55%)7,265.98 (97.43%)7,568.04 (98.55%)✅ (view plot)8.00 (-16.94%)5.60 (69.97%)13.66 (58.55%)✅ (view plot)138.00 (+1.10%)131.23 (95.09%)141.78 (97.33%)
serialize_deserialize_authorize✅ (view plot)24,931.00 (+1.47%)24,195.18 (97.05%)24,945.85 (99.94%)✅ (view plot)9,920.00 (+0.22%)9,770.58 (98.49%)10,026.31 (98.94%)✅ (view plot)14,016.00 (+0.33%)13,782.21 (98.33%)14,156.39 (99.01%)✅ (view plot)34.00 (-5.22%)29.48 (86.71%)42.27 (80.44%)✅ (view plot)307.00 (+3.10%)288.40 (93.94%)307.13 (99.96%)
serialize_deserialize_handle_authorize✅ (view plot)30,435.00 (+0.69%)29,896.67 (98.23%)30,556.32 (99.60%)✅ (view plot)12,097.00 (+0.04%)11,979.05 (99.02%)12,205.16 (99.11%)✅ (view plot)17,140.00 (+0.13%)16,960.25 (98.95%)17,275.75 (99.21%)✅ (view plot)55.00 (-4.28%)50.12 (91.12%)64.80 (84.88%)✅ (view plot)372.00 (+1.55%)357.56 (96.12%)375.08 (99.18%)
serialize_deserialize_handle_submit✅ (view plot)126,497.00 (+0.04%)125,859.46 (99.50%)127,040.75 (99.57%)✅ (view plot)73,363.00 (+0.13%)72,942.26 (99.43%)73,594.89 (99.68%)✅ (view plot)105,197.00 (+0.16%)104,561.43 (99.40%)105,497.81 (99.71%)✅ (view plot)109.00 (-3.17%)89.57 (82.18%)135.57 (80.40%)✅ (view plot)593.00 (-0.49%)589.35 (99.38%)602.51 (98.42%)
serialize_deserialize_handle_subscribe✅ (view plot)28,009.00 (+1.35%)27,030.77 (96.51%)28,241.17 (99.18%)✅ (view plot)9,666.00 (+0.25%)9,532.86 (98.62%)9,751.10 (99.13%)✅ (view plot)13,679.00 (+0.30%)13,487.21 (98.60%)13,789.10 (99.20%)✅ (view plot)59.00 (-8.67%)55.98 (94.88%)73.22 (80.57%)✅ (view plot)401.00 (+2.63%)374.89 (93.49%)406.52 (98.64%)
serialize_deserialize_submit✅ (view plot)115,445.00 (+0.25%)114,590.43 (99.26%)115,725.16 (99.76%)✅ (view plot)68,223.00 (+0.21%)67,768.21 (99.33%)68,391.77 (99.75%)✅ (view plot)97,935.00 (+0.25%)97,229.80 (99.28%)98,155.79 (99.78%)✅ (view plot)65.00 (-0.27%)52.30 (80.47%)78.05 (83.28%)✅ (view plot)491.00 (+0.27%)483.13 (98.40%)496.25 (98.94%)
serialize_deserialize_subscribe✅ (view plot)23,469.00 (+1.77%)22,460.07 (95.70%)23,661.34 (99.19%)✅ (view plot)8,225.00 (+0.35%)8,086.42 (98.32%)8,306.57 (99.02%)✅ (view plot)11,589.00 (+0.38%)11,394.00 (98.32%)11,696.64 (99.08%)✅ (view plot)38.00 (-0.71%)32.87 (86.50%)43.68 (87.01%)✅ (view plot)334.00 (+3.23%)308.36 (92.32%)338.73 (98.60%)
serialize_submit✅ (view plot)99,835.00 (-0.06%)99,342.24 (99.51%)100,438.40 (99.40%)✅ (view plot)61,566.00 (+0.08%)61,241.34 (99.47%)61,787.60 (99.64%)✅ (view plot)88,350.00 (+0.10%)87,872.39 (99.46%)88,642.53 (99.67%)✅ (view plot)50.00 (-1.76%)36.55 (73.11%)65.23 (76.65%)✅ (view plot)321.00 (-1.26%)318.54 (99.23%)331.66 (96.79%)
serialize_subscribe✅ (view plot)11,405.00 (+0.25%)11,070.71 (97.07%)11,683.52 (97.62%)✅ (view plot)4,195.00 (+0.31%)4,076.34 (97.17%)4,287.37 (97.85%)✅ (view plot)5,840.00 (+0.35%)5,678.45 (97.23%)5,960.45 (97.98%)✅ (view plot)14.00 (-3.03%)8.73 (62.33%)20.15 (69.49%)✅ (view plot)157.00 (+0.17%)150.74 (96.02%)162.71 (96.49%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

Copy link
Contributor

github-actions bot commented Aug 14, 2024

🐰Bencher

ReportTue, September 3, 2024 at 17:06:33 UTC
ProjectStratum V2 (SRI)
Branch1099/merge
Testbedsv1
Click to view all benchmark results
BenchmarkLatencyLatency Results
nanoseconds (ns) | (Δ%)
Latency Lower Boundary
nanoseconds (ns) | (%)
Latency Upper Boundary
nanoseconds (ns) | (%)
client-submit-serialize✅ (view plot)7,026.30 (+2.60%)6,340.01 (90.23%)7,356.36 (95.51%)
client-submit-serialize-deserialize✅ (view plot)8,180.90 (+4.82%)7,227.86 (88.35%)8,381.23 (97.61%)
client-submit-serialize-deserialize-handle/client-submit-serialize-deserialize-handle✅ (view plot)8,723.30 (+4.11%)7,887.15 (90.41%)8,870.02 (98.35%)
client-sv1-authorize-serialize-deserialize-handle/client-sv1-authorize-serialize-deserialize-handle✅ (view plot)925.97 (+2.09%)863.16 (93.22%)950.82 (97.39%)
client-sv1-authorize-serialize-deserialize/client-sv1-authorize-serialize-deserialize✅ (view plot)710.27 (+0.66%)671.71 (94.57%)739.58 (96.04%)
client-sv1-authorize-serialize/client-sv1-authorize-serialize✅ (view plot)249.47 (+0.12%)241.45 (96.78%)256.88 (97.12%)
client-sv1-get-authorize/client-sv1-get-authorize✅ (view plot)158.08 (+0.44%)151.68 (95.95%)163.08 (96.93%)
client-sv1-get-submit✅ (view plot)6,872.90 (+3.70%)6,135.24 (89.27%)7,120.43 (96.52%)
client-sv1-get-subscribe/client-sv1-get-subscribe✅ (view plot)280.18 (+1.12%)260.27 (92.89%)293.87 (95.34%)
client-sv1-subscribe-serialize-deserialize-handle/client-sv1-subscribe-serialize-deserialize-handle✅ (view plot)789.34 (+4.21%)713.78 (90.43%)801.11 (98.53%)
client-sv1-subscribe-serialize-deserialize/client-sv1-subscribe-serialize-deserialize✅ (view plot)637.95 (+3.06%)586.63 (91.95%)651.34 (97.94%)
client-sv1-subscribe-serialize/client-sv1-subscribe-serialize✅ (view plot)199.34 (-3.95%)194.43 (97.53%)220.66 (90.34%)

Bencher - Continuous Benchmarking
View Public Perf Page
Docs | Repo | Chat | Help

@pavlenex
Copy link
Collaborator

pavlenex commented Aug 14, 2024

I did get a flaky test, but it wasn't on what happened on #1001 and #1055 so I re-run it and it seems adding a timer did a trick. Good quick fix ✅

I did rebase your PR to run the tests in the first place, but it created a merge commit, not sure how @jbesraa feels about this, but you can probably squash it. Sorrry about that.

@plebhash plebhash changed the base branch from dev to main August 28, 2024 19:21
@plebhash plebhash added the ready-to-be-merged triggers auto rebase bot label Sep 3, 2024
@GitGab19 GitGab19 merged commit aba4182 into stratum-mining:main Sep 4, 2024
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-be-merged triggers auto rebase bot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Prevent infinite loop in SV1-Test when the SV2 Pool fails to connect to the template provider
4 participants