H.264 error resiliency

Tasks

• to prevent running out of the bitrate, H.264 frames must be as even as possible
• let decoder process even partially corrupted frames
  • test which decoders crash on scrambled data
  • zero buffer space where data are missing due to missing packets (maybe not needed but more safe)
• divide RTP packets according to H.264 NAL units (like it is already for RFC-compliant transmission)
• consider fetching the lavd not H.264 frames but NAL units/packets
• consider invalidating incomplete NAL units
• different transport (SRT/QUIC/UDT)

Testing

• testing data for all tests trud:/mnt/xfs/videa/New/Zeland/NZ-1080p-yuv-H.264-15M
• loss is emulated by https://wiki.linuxfoundation.org/networking/netem (works on egress)

Test Case 1, 2

First test without and second with Reed-Solomon forward error correction:

sudo tc qdisc replace dev lo root netem loss 5%
uv --playback NZ-1080p-yuv-H.264-15M -d gl:fs [-f rs:200:220]

Test results

• whereas passing incomplete frame gains better resiliency for H.264 when no FEC is deployed, it doesn't with RS. This may be probably that there is too much lost in order to decoder to be able to go over.
• passing to the H264 decoder only valid frames doesn't improve situation (ref here)