feat: TOS improvements

[larseggert]

Broken out of #1678 to reduce the size of that PR.

[mxinden]

One comment. Otherwise looks good to me.

[github-actions]

Benchmark results

Performance differences relative to 57dd4ec.

• drain a timer quickly time: [390.96 ns 398.90 ns 406.56 ns]
  change: [-1.4487% +0.1214% +1.6284%] (p = 0.89 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 1+1 entries
  time: [194.51 ns 194.90 ns 195.31 ns]
  change: [-0.1573% +0.1984% +0.6059%] (p = 0.29 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 3+1 entries
  time: [234.74 ns 235.31 ns 235.92 ns]
  change: [-0.5128% -0.0017% +0.5595%] (p = 0.99 > 0.05)
  No change in performance detected.

• coalesce_acked_from_zero 10+1 entries
  time: [233.79 ns 234.30 ns 234.97 ns]
  change: [-1.1163% -0.5821% -0.1005%] (p = 0.03 < 0.05)
  Change within noise threshold.

• coalesce_acked_from_zero 1000+1 entries
  time: [213.03 ns 213.26 ns 213.53 ns]
  change: [-0.8219% -0.1751% +0.4809%] (p = 0.63 > 0.05)
  No change in performance detected.

• RxStreamOrderer::inbound_frame()
  time: [117.79 ms 117.86 ms 117.93 ms]
  change: [-0.0953% -0.0043% +0.0836%] (p = 0.93 > 0.05)
  No change in performance detected.

• transfer/Run multiple transfers with varying seeds
  time: [118.02 ms 118.28 ms 118.54 ms]
  thrpt: [33.744 MiB/s 33.819 MiB/s 33.893 MiB/s]
  change:
  time: [+0.0631% +0.3847% +0.7016%] (p = 0.02 < 0.05)
  thrpt: [-0.6967% -0.3832% -0.0630%]
  Change within noise threshold.

• transfer/Run multiple transfers with the same seed
  time: [119.09 ms 119.27 ms 119.44 ms]
  thrpt: [33.488 MiB/s 33.538 MiB/s 33.588 MiB/s]
  change:
  time: [+0.1037% +0.3015% +0.5047%] (p = 0.00 < 0.05)
  thrpt: [-0.5022% -0.3006% -0.1036%]
  Change within noise threshold.

Client/server transfer results

Transfer of 134217728 bytes over loopback.

Client	Server	CC	Pacing	Mean [ms]	Min [ms]	Max [ms]	Relative
msquic	msquic			721.8 ± 329.6	381.8	1369.1	1.00
neqo	msquic	reno	on	2022.6 ± 209.5	1844.4	2491.1	1.00
neqo	msquic	reno		2079.4 ± 202.7	1912.5	2458.2	1.00
neqo	msquic	cubic	on	2085.5 ± 254.3	1864.9	2467.7	1.00
neqo	msquic	cubic		2109.1 ± 297.2	1856.7	2597.8	1.00
msquic	neqo	reno	on	4518.1 ± 340.9	4121.1	5105.7	1.00
msquic	neqo	reno		4450.5 ± 184.5	4256.1	4859.6	1.00
msquic	neqo	cubic	on	4507.9 ± 182.4	4220.6	4717.6	1.00
msquic	neqo	cubic		4511.6 ± 337.5	4178.7	5176.0	1.00
neqo	neqo	reno	on	3950.4 ± 323.4	3537.3	4635.4	1.00
neqo	neqo	reno		3739.3 ± 300.4	3281.0	4416.6	1.00
neqo	neqo	cubic	on	4353.0 ± 362.6	3581.4	4882.6	1.00
neqo	neqo	cubic		4250.5 ± 390.5	3286.6	4741.9	1.00

⬇️ Download logs