WebRTC audio not working with OPUS/1

[felipecrs]

Again, trying to make WebRTC work with OPUS audio from Thingino.

streams:
  terreiro:
  - rtsp://thingino:[email protected]/ch1

Maybe it's because it's OPUS/1 not OPUS/2? Maybe it's the same reason why #1505 is happening.

{
  "producers": [
    {
      "id": 14,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "192.168.1.39:554",
      "url": "rtsp://thingino:[email protected]/ch1",
      "sdp": "v=0\r\no=- 1733960107351613 1 IN IP4 192.168.1.39\r\ns=thingino prudynt\r\ni=stream1\r\nt=0 0\r\na=tool:LIVE555 Streaming Media v2024.11.28\r\na=type:broadcast\r\na=control:*\r\na=range:npt=now-\r\na=x-qt-text-nam:thingino prudynt\r\na=x-qt-text-inf:stream1\r\nm=video 0 RTP/AVP 96\r\nc=IN IP4 0.0.0.0\r\nb=AS:5000\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=64001E;sprop-parameter-sets=Z2QAHqw7UFAX/LCAAAADAIAAABlC,aO48gA==\r\na=control:track1\r\nm=audio 0 RTP/AVP 97\r\nc=IN IP4 0.0.0.0\r\nb=AS:32\r\na=rtpmap:97 OPUS/48000\r\na=control:track2\r\n",
      "user_agent": "go2rtc/1.9.7",
      "medias": [
        "video, recvonly, H264",
        "audio, recvonly, OPUS/48000"
      ],
      "receivers": [
        {
          "id": 20,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 30,
            "profile": "High"
          },
          "childs": [
            87,
            169
          ],
          "bytes": 28946303,
          "packets": 24551
        },
        {
          "id": 62,
          "codec": {
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "childs": [
            63,
            66
          ],
          "bytes": 5499651,
          "packets": 34589
        }
      ],
      "bytes_recv": 35188634
    }
  ],
  "consumers": [
    {
      "id": 56,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:36056",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.7\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 OPUS/48000\r\na=control:trackID=0\r\n",
      "user_agent": "go2rtc/1.9.7",
      "medias": [
        "audio, sendonly, ALL"
      ],
      "senders": [
        {
          "id": 63,
          "codec": {
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "parent": 62,
          "bytes": 5499651,
          "packets": 34589
        }
      ],
      "bytes_send": 6053075
    },
    {
      "id": 61,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:36084",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.7\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=audio 0 RTP/AVP 96\r\na=rtpmap:96 OPUS/48000\r\na=control:trackID=0\r\n",
      "user_agent": "FFmpeg Frigate/0.14.1-f4f3cfa",
      "medias": [
        "audio, sendonly, OPUS"
      ],
      "senders": [
        {
          "id": 66,
          "codec": {
            "codec_name": "opus",
            "codec_type": "audio",
            "sample_rate": 48000
          },
          "parent": 62,
          "bytes": 5499651,
          "packets": 34589
        }
      ],
      "bytes_send": 6053075
    },
    {
      "id": 86,
      "format_name": "rtsp",
      "protocol": "rtsp+tcp",
      "remote_addr": "127.0.0.1:45040",
      "sdp": "v=0\r\no=- 1 1 IN IP4 0.0.0.0\r\ns=go2rtc/1.9.7\r\nc=IN IP4 0.0.0.0\r\nt=0 0\r\nm=video 0 RTP/AVP 96\r\na=rtpmap:96 H264/90000\r\na=fmtp:96 packetization-mode=1;profile-level-id=64001E;sprop-parameter-sets=Z2QAHqw7UFAX/LCAAAADAIAAABlC,aO48gA==\r\na=control:trackID=0\r\n",
      "user_agent": "FFmpeg Frigate/0.14.1-f4f3cfa",
      "medias": [
        "video, sendonly, ANY"
      ],
      "senders": [
        {
          "id": 87,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video",
            "level": 30,
            "profile": "High"
          },
          "parent": 20,
          "bytes": 28301630,
          "packets": 24015
        }
      ],
      "bytes_send": 28685870
    },
    {
      "id": 168,
      "format_name": "webrtc",
      "protocol": "ws+udp",
      "remote_addr": "192.168.1.15:49875 host",
      "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36",
      "medias": [
        "video, sendonly, VP8, VP9, H264, AV1",
        "audio, sendonly, OPUS/48000/2, G722/8000, PCMU/8000, PCMA/8000, L16, PCML"
      ],
      "senders": [
        {
          "id": 169,
          "codec": {
            "codec_name": "h264",
            "codec_type": "video"
          },
          "parent": 20,
          "bytes": 5764018,
          "packets": 4851
        }
      ],
      "bytes_send": 5833856
    }
  ]
}

[felipecrs]

PS: MSE works.

[AlexxIT]

I have never saw OPUS/48000 without /2. I don't think that's the standard at all. It needs to be further investigated.

[felipecrs]

As Thingino is open source, I can try to change it to output in OPUS/48000/2.

I really cannot comment whether this is a standard or not, but I guess it makes sense to send a single audio track given there is a single microphone. Two audio tracks would use more data, I suppose, for no good reason.

What is your opinion about it? Do you think I should try to patch Thingino?

[miguelangel-nubla]

I am also seeing this on a OpenIPC camera, https://opus-codec.org/ list mono and stereo as part of the specification.
Indicates to just duplicate the channel in this situation.

[AlexxIT]

Of course OPUS can support mono any other bitrates. But I'm not sure about OPUS in the RTSP:

https://datatracker.ietf.org/doc/html/rfc7587

 Example 1: Standard mono session with 48000 Hz clock rate

       m=audio 54312 RTP/AVP 101
       a=rtpmap:101 opus/48000/2

[miguelangel-nubla]

You are right:
https://datatracker.ietf.org/doc/html/rfc7587#section-7

   o  The media subtype ("opus") goes in SDP "a=rtpmap" as the encoding
      name.  The RTP clock rate in "a=rtpmap" MUST be 48000, and the
      number of channels MUST be 2.

So Thingino and OpenIPC are currently non compliant.

Additionally, since opus/48000/2 is the only valid definition, maybe makes sense to regexp replace all opus to 48000/2 in the incoming SDP and avoid future issues with non compliant servers.

[felipecrs]

Additionally, since opus/48000/2 is the only valid definition, maybe makes sense to regexp replace all opus to 48000/2 in the incoming SDP and avoid future issues with non compliant servers.

Do you think that would be enough to make it work?

[gtxaspec]

I've manually adjusted this in prudynt to opus/2 and it worked with go2rtc webrtc. Only in internal testing, not production. Uncertain if it will cause problems with other clients though.