Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ci: More fixes to eventually generate a diff for QNS results #1867

Merged
merged 2 commits into from
May 2, 2024
Merged

ci: More fixes to eventually generate a diff for QNS results #1867

merged 2 commits into from
May 2, 2024

Conversation

larseggert
Copy link
Collaborator

WIP

@github-actions GitHub Actions
Copy link

github-actions bot commented May 2, 2024 •
edited
Loading

Failed Interop Tests

QUIC Interop Runner, client vs. server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

@github-actions GitHub Actions
Copy link

github-actions bot commented May 2, 2024 •
edited
Loading

Benchmark results

Performance differences relative to 7070e7b.

  • drain a timer quickly time: [310.01 ns 318.49 ns 326.46 ns]
    change: [-1.9586% +0.3007% +2.5837%] (p = 0.79 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 1+1 entries
    time: [194.35 ns 194.89 ns 195.43 ns]
    change: [+0.1564% +0.6558% +1.2481%] (p = 0.02 < 0.05)
    Change within noise threshold.

  • coalesce_acked_from_zero 3+1 entries
    time: [236.97 ns 237.57 ns 238.21 ns]
    change: [-0.4035% +0.0992% +0.5474%] (p = 0.70 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 10+1 entries
    time: [236.02 ns 236.94 ns 237.98 ns]
    change: [-0.9947% -0.2489% +0.3826%] (p = 0.51 > 0.05)
    No change in performance detected.

  • coalesce_acked_from_zero 1000+1 entries
    time: [217.25 ns 218.55 ns 221.27 ns]
    change: [-0.4078% +14.797% +43.667%] (p = 0.31 > 0.05)
    No change in performance detected.

  • RxStreamOrderer::inbound_frame()
    time: [118.12 ms 118.21 ms 118.31 ms]
    change: [-3.5051% -3.4102% -3.3198%] (p = 0.00 < 0.05)
    💚 Performance has improved.

  • transfer/Run multiple transfers with varying seeds
    time: [117.99 ms 118.29 ms 118.59 ms]
    thrpt: [33.730 MiB/s 33.815 MiB/s 33.902 MiB/s]
    change:
    time: [+0.1950% +0.5033% +0.8290%] (p = 0.00 < 0.05)
    thrpt: [-0.8222% -0.5008% -0.1946%]
    Change within noise threshold.

  • transfer/Run multiple transfers with the same seed
    time: [118.06 ms 118.29 ms 118.51 ms]
    thrpt: [33.751 MiB/s 33.816 MiB/s 33.880 MiB/s]
    change:
    time: [+0.2335% +0.4897% +0.7551%] (p = 0.00 < 0.05)
    thrpt: [-0.7494% -0.4873% -0.2330%]
    Change within noise threshold.

  • 1-conn/1-100mb-resp (aka. Download)/client
    time: [1.1112 s 1.1183 s 1.1273 s]
    thrpt: [88.710 MiB/s 89.422 MiB/s 89.995 MiB/s]
    change:
    time: [-0.0438% +1.9295% +3.5391%] (p = 0.05 > 0.05)
    thrpt: [-3.4181% -1.8930% +0.0438%]
    No change in performance detected.

  • 1-conn/10_000-parallel-1b-resp (aka. RPS)/client
    time: [427.16 ms 429.48 ms 431.80 ms]
    thrpt: [23.159 Kelem/s 23.284 Kelem/s 23.410 Kelem/s]
    change:
    time: [-0.3718% +0.4094% +1.1964%] (p = 0.32 > 0.05)
    thrpt: [-1.1823% -0.4077% +0.3732%]
    No change in performance detected.

  • 1-conn/1-1b-resp (aka. HPS)/client
    time: [48.916 ms 49.296 ms 49.671 ms]
    thrpt: [20.132 elem/s 20.286 elem/s 20.443 elem/s]
    change:
    time: [+0.1278% +1.1881% +2.2877%] (p = 0.03 < 0.05)
    thrpt: [-2.2366% -1.1741% -0.1276%]
    Change within noise threshold.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

Client Server CC Pacing Mean [ms] Min [ms] Max [ms] Relative
msquic msquic 812.9 ± 246.8 405.9 1132.6 1.00
neqo msquic reno on 941.8 ± 320.9 747.2 1574.3 1.00
neqo msquic reno 997.1 ± 273.4 777.6 1390.8 1.00
neqo msquic cubic on 959.2 ± 178.2 756.5 1289.3 1.00
neqo msquic cubic 999.9 ± 206.8 797.1 1329.3 1.00
msquic neqo reno on 4418.7 ± 279.1 4014.9 5054.5 1.00
msquic neqo reno 4352.2 ± 362.5 4067.1 5322.9 1.00
msquic neqo cubic on 4470.1 ± 242.6 4174.4 4940.8 1.00
msquic neqo cubic 4436.6 ± 286.3 4116.9 5005.9 1.00
neqo neqo reno on 3539.2 ± 357.7 2738.7 4166.5 1.00
neqo neqo reno 3319.4 ± 187.4 3084.4 3685.3 1.00
neqo neqo cubic on 4239.3 ± 363.7 3535.7 4986.1 1.00
neqo neqo cubic 4054.3 ± 464.4 2879.9 4447.5 1.00

⬇️ Download logs

@codecov Codecov
Copy link

codecov bot commented May 2, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 93.25%. Comparing base (7070e7b) to head (92fd589).

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1867   +/-   ##
=======================================
  Coverage   93.25%   93.25%           
=======================================
  Files         110      110           
  Lines       35813    35813           
=======================================
  Hits        33397    33397           
  Misses       2416     2416

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@larseggert larseggert changed the title ci: Generate a diff for QNS results ci: More fixes to eventually generate a diff for QNS results May 2, 2024
@larseggert larseggert marked this pull request as ready for review May 2, 2024 12:38
@larseggert larseggert merged commit 8192300 into mozilla:main May 2, 2024
50 checks passed
@larseggert larseggert deleted the ci-qns-diff-cont branch May 2, 2024 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KershawChang KershawChang Awaiting requested review from KershawChang KershawChang is a code owner

@martinthomson martinthomson Awaiting requested review from martinthomson martinthomson is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@larseggert