neqo v0.7.6

[KershawChang]

Neqo v0.7.6

[github-actions]

Failed Interop Tests

QUIC Interop Runner, client vs. server

• aioquic vs. neqo-latest: R A
• chrome vs. neqo-latest: 3
• go-x-net vs. neqo-latest: U A
• kwik vs. neqo-latest: A
• lsquic vs. neqo-latest: R A
• msquic vs. neqo-latest: Z A L1
• mvfst vs. neqo-latest: Z 3 A L1 C1
• neqo vs. neqo-latest: Z A
• neqo-latest vs. aioquic: Z L1 C1
• neqo-latest vs. lsquic: C1
• neqo-latest vs. msquic: Z A L1 C1
• neqo-latest vs. mvfst: DC B U A L1 L2 C1 C2
• neqo-latest vs. neqo: Z A
• neqo-latest vs. neqo-latest: Z A
• neqo-latest vs. ngtcp2: Z
• neqo-latest vs. quic-go: L1
• neqo-latest vs. quiche: L1 C1
• neqo-latest vs. quinn: E A
• neqo-latest vs. s2n-quic: R C1
• neqo-latest vs. xquic: A
• ngtcp2 vs. neqo-latest: C20 A
• picoquic vs. neqo-latest: R A
• quic-go vs. neqo-latest: A
• quiche vs. neqo-latest: R 3 A
• quinn vs. neqo-latest: R Z E A
• s2n-quic vs. neqo-latest: E A
• xquic vs. neqo-latest: M A

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

• aioquic vs. neqo-latest: H DC LR C20 M S Z 3 B L1 L2 C1 C2 6
• go-x-net vs. neqo-latest: H DC LR M B L2 C2 6
• kwik vs. neqo-latest: H DC LR C20 M S R Z 3 B U L1 L2 C1 C2 6 V2
• lsquic vs. neqo-latest: H DC LR M S 3 B E L1 L2 C1 C2 6 V2
• msquic vs. neqo-latest: H DC LR C20 M S R B U L2 C1 C2 6 V2
• mvfst vs. neqo-latest: H DC LR M B L2 C2 6
• neqo vs. neqo-latest: H DC LR C20 M S R 3 B U E L1 L2 C1 C2 6 V2
• neqo-latest vs. aioquic: H DC LR C20 M S R 3 B U A L2 C2 6
• neqo-latest vs. go-x-net: H DC LR M B U A L2 C2 6
• neqo-latest vs. haproxy: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2
• neqo-latest vs. kwik: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6 V2
• neqo-latest vs. lsquic: H DC LR C20 M S R Z 3 B U E A L1 L2 C2 6 V2
• neqo-latest vs. msquic: H DC LR C20 M S R B U L2 C2 6 V2
• neqo-latest vs. mvfst: H LR M R Z 3 6
• neqo-latest vs. neqo: H DC LR C20 M S R 3 B U E L1 L2 C1 C2 6 V2
• neqo-latest vs. neqo-latest: H DC LR C20 M S R 3 B U E L1 L2 C1 C2 6 V2
• neqo-latest vs. nginx: H DC LR C20 M S R Z 3 B U A L1 L2 C1 C2 6
• neqo-latest vs. ngtcp2: H DC LR C20 M S R 3 B U E A L1 L2 C1 C2 6 V2
• neqo-latest vs. picoquic: H DC LR C20 M S R Z 3 B U E A L1 L2 C1 C2 6 V2
• neqo-latest vs. quic-go: H DC LR C20 M S R Z 3 B U A L2 C1 C2 6
• neqo-latest vs. quiche: H DC LR C20 M S R Z 3 B U A L2 C2 6
• neqo-latest vs. quinn: H DC LR C20 M S R Z 3 B U L2 C2 6
• neqo-latest vs. s2n-quic: H DC LR C20 M S 3 B U E A L1 L2 C2 6
• neqo-latest vs. xquic: H DC LR C20 M R Z 3 B U L1 L2 C1 C2 6
• ngtcp2 vs. neqo-latest: H DC LR M S R Z 3 B U E L1 L2 C1 C2 6 V2
• picoquic vs. neqo-latest: H DC LR C20 M S Z 3 B U E L1 L2 C1 C2 6 V2
• quic-go vs. neqo-latest: H DC LR C20 M S R Z 3 B U L1 L2 C1 C2 6
• quiche vs. neqo-latest: H DC LR M S Z B L1 L2 C1 C2 6
• quinn vs. neqo-latest: H DC LR C20 M S 3 B U L2 C2 6
• s2n-quic vs. neqo-latest: H DC LR M S R 3 B L1 L2 C1 C2 6
• xquic vs. neqo-latest: H DC LR C20 S R Z 3 B U L1 L2 C1 C2 6

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

• aioquic vs. neqo-latest: U E V2
• chrome vs. neqo-latest: H DC LR C20 M S R Z B U E A L1 L2 C1 C2 6 V2
• go-x-net vs. neqo-latest: C20 S R Z 3 E L1 C1 V2
• kwik vs. neqo-latest: E
• lsquic vs. neqo-latest: C20 Z U
• msquic vs. neqo-latest: 3 E
• mvfst vs. neqo-latest: C20 S R U E V2
• neqo-latest vs. aioquic: E V2
• neqo-latest vs. go-x-net: C20 S R Z 3 E L1 C1 V2
• neqo-latest vs. haproxy: E
• neqo-latest vs. kwik: E
• neqo-latest vs. msquic: 3 E
• neqo-latest vs. mvfst: C20 S E V2
• neqo-latest vs. nginx: E V2
• neqo-latest vs. quic-go: E V2
• neqo-latest vs. quiche: E V2
• neqo-latest vs. quinn: L1 C1 V2
• neqo-latest vs. s2n-quic: Z V2
• neqo-latest vs. xquic: S E V2
• quic-go vs. neqo-latest: E V2
• quiche vs. neqo-latest: C20 U E V2
• quinn vs. neqo-latest: L1 C1 V2
• s2n-quic vs. neqo-latest: C20 Z U V2
• xquic vs. neqo-latest: E V2

[github-actions]

Benchmark results

Performance differences relative to 8c4411a.

• drain a timer quickly time: [310.52 ns 317.79 ns 324.57 ns]
  change: [-1.9744% +0.2488% +2.6312%] (p = 0.83 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 1+1 entries
  time: [198.14 ns 198.58 ns 199.05 ns]
  change: [+0.8738% +1.6244% +2.1704%] (p = 0.00 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 3+1 entries
  time: [240.79 ns 241.31 ns 241.87 ns]
  change: [+1.2253% +1.8740% +2.8342%] (p = 0.00 < 0.05)
  💔 Performance has regressed.

• coalesce_acked_from_zero 10+1 entries
  time: [240.21 ns 241.16 ns 242.23 ns]
  change: [+1.0687% +1.6143% +2.0958%] (p = 0.00 < 0.05)
  💔 Performance has regressed.

• coalesce_acked_from_zero 1000+1 entries
  time: [221.44 ns 221.66 ns 221.88 ns]
  change: [+0.9155% +1.5640% +2.2193%] (p = 0.00 < 0.05)
  Change within noise threshold.

• RxStreamOrderer::inbound_frame()
  time: [118.67 ms 118.74 ms 118.81 ms]
  change: [-0.7200% -0.6304% -0.5432%] (p = 0.00 < 0.05)
  Change within noise threshold.

• transfer/Run multiple transfers with varying seeds
  time: [118.32 ms 118.58 ms 118.84 ms]
  thrpt: [33.659 MiB/s 33.732 MiB/s 33.807 MiB/s]
  change:
  time: [-0.7236% -0.4128% -0.1259%] (p = 0.01 < 0.05)
  thrpt: [+0.1261% +0.4146% +0.7289%]
  Change within noise threshold.

• transfer/Run multiple transfers with the same seed
  time: [118.16 ms 118.37 ms 118.57 ms]
  thrpt: [33.736 MiB/s 33.793 MiB/s 33.852 MiB/s]
  change:
  time: [-0.6558% -0.3937% -0.1353%] (p = 0.00 < 0.05)
  thrpt: [+0.1355% +0.3952% +0.6601%]
  Change within noise threshold.

• 1-conn/1-100mb-resp (aka. Download)/client
  time: [1.1463 s 1.1520 s 1.1571 s]
  thrpt: [86.424 MiB/s 86.804 MiB/s 87.236 MiB/s]
  change:
  time: [+0.3589% +1.6165% +2.6391%] (p = 0.02 < 0.05)
  thrpt: [-2.5712% -1.5908% -0.3576%]
  Change within noise threshold.

• 1-conn/10_000-parallel-1b-resp (aka. RPS)/client
  time: [432.05 ms 434.23 ms 436.42 ms]
  thrpt: [22.914 Kelem/s 23.029 Kelem/s 23.146 Kelem/s]
  change:
  time: [-0.3855% +0.3587% +1.0881%] (p = 0.33 > 0.05)
  thrpt: [-1.0764% -0.3574% +0.3870%]
  No change in performance detected.

• 1-conn/1-1b-resp (aka. HPS)/client
  time: [48.047 ms 48.458 ms 48.847 ms]
  thrpt: [20.472 elem/s 20.636 elem/s 20.813 elem/s]
  change:
  time: [-2.0647% -0.3574% +1.2240%] (p = 0.67 > 0.05)
  thrpt: [-1.2092% +0.3587% +2.1082%]
  No change in performance detected.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

Client	Server	CC	Pacing	Mean [ms]	Min [ms]	Max [ms]	Relative
msquic	msquic			744.0 ± 331.8	369.2	1276.6	1.00
neqo	msquic	reno	on	871.6 ± 196.8	764.1	1415.3	1.00
neqo	msquic	reno		902.7 ± 224.9	760.9	1459.4	1.00
neqo	msquic	cubic	on	854.1 ± 122.1	751.9	1082.8	1.00
neqo	msquic	cubic		930.9 ± 203.1	750.2	1368.3	1.00
msquic	neqo	reno	on	4372.8 ± 252.3	3989.9	4887.7	1.00
msquic	neqo	reno		4330.6 ± 265.3	4059.4	4988.0	1.00
msquic	neqo	cubic	on	4448.1 ± 299.6	4168.9	5175.8	1.00
msquic	neqo	cubic		4397.3 ± 269.5	4103.1	5024.3	1.00
neqo	neqo	reno	on	3479.3 ± 275.5	2848.5	3819.8	1.00
neqo	neqo	reno		3723.3 ± 188.4	3332.0	4039.3	1.00
neqo	neqo	cubic	on	4102.7 ± 428.1	2971.8	4493.6	1.00
neqo	neqo	cubic		4003.4 ± 479.6	2698.6	4343.8	1.00

⬇️ Download logs