Introducing SRT Publisher

[dimiden]

🎉 Introducing SRT Publisher for OvenMediaEngine! 🎉

We are excited to announce that the development of the SRT Publisher is now complete.
While there are still some areas for improvement, the core functionality is ready for use, and we are opening it up to the community.

[Areas for Improvement]

1. Refining the streamid Format
   • The streamid in the SRT provider is currently implemented as a full URL, which often results in excessively long SRT URLs.
   • For the SRT Publisher, we have adopted a more concise and intuitive format: vhost/app/stream.
   • If this approach proves effective, we plan to update the SRT provider to use the same format in the future.
2. Handling Multiple Audio/Video Tracks
   • Currently, when a stream contains multiple audio or video tracks, the first track is selected by default.
   • We are considering ways to allow better track mapping in such cases.
     • For example, LLHLS addresses this by specifying tracks in OutputProfile.Playlist.

[Learn More]
For detailed instructions on how to use the SRT Publisher, please refer to the manual: https://airensoft.gitbook.io/ovenmediaengine/dev/streaming/srt

We hope you will try out the new SRT Publisher feature and share your feedback. Your input will help us refine and enhance this functionality further!
Wishing you a Happy New Year and looking forward to your thoughts and suggestions!

[bchah]

@dimiden Thank you so much for this! I tested it just now and have some notes:

1. SRT Publisher works fine with RTMP provider, however using it with SRT provider causes an issue when the original stream contains a DATA track:

[2025-01-05 19:17:51.861] E [OutboundWorker:754] MPEG-2 TS | mpegts_packetizer.cpp:64   | Data Track (Unknown bitstream format) is not supported
[2025-01-05 19:17:51.861] E [OutboundWorker:754] SRTPublisher | srt_stream.cpp:124  | [#default#live/helloWorld(1260719847)] Failed to add track to packetizer
[2025-01-05 19:18:30.328] I [SPSRTPub-s9998:734] Monitor | stream_metrics.cpp:114  | A new session has started playing #default#live/helloWorld on the SRT publisher. SRT(1)/Stream total(1)/App total(1)

In this case, the client connects OK but the stream does not actually play back on the client. OBS seems to include a data track by default and I know many encoders include timecode on the data track, so this will need to be addressed before this can be widely implemented with SRT Provider supported.

2. This could be related to the data track issue but I also noticed that HEVC is excluded in your documentation. However HEVC is fully supported by the SRT provider and SRT push publisher, so it would only make sense that the SRT publisher supports it too.

3. In your example for enabling SRT Publisher, it references <Provider> where I think you meant to reference <Publisher>.

Thank you again for your hard work! ⭐️

[dimiden]

@bchah
Thank you for the feedback!
I have just fixed the typo and modified it so that the data channel ignores unsupported bitstreams for now.

Could you let me know how you included the data track in SRT? If I can reproduce the problem, I would like to review whether improvements are possible.

[bchah]

@dimiden Thank you and I will test that change ASAP. I think I found a clue about the data track. Here is the stream info:

[Stream Info]
id(2479206773), msid(0), output(helloWorld), SourceType(Transcoder), RepresentationType(Source), Created Time (Mon Jan  6 03:06:18 2025) UUID(955ed55d-00ed-405a-87f2-a1cb66223c2c/default/#default#live/helloWorld/o)
	>> Origin Stream Info
	id(100), output(helloWorld), SourceType(SRT), Created Time (Mon Jan  6 03:06:17 2025)

	Data  Track #0: Public Name(Data_1) Variant Name(Data) Codec(0,Unknown,Passthrough) BSF(Unknown) timebase(1/1000)
	Video Track #1: Public Name(Video_256) Variant Name(original_video) Bitrate(1.02Mb) Codec(1,H264,Passthrough:0) BSF(H264_ANNEXB) Resolution(960x540) Framerate(30.48) KeyInterval(55/frame) SkipFrames(-1) BFrames(0) timebase(1/90000)
	Audio Track #2: Public Name(Audio_257) Variant Name(aac_audio) Bitrate(128.00Kb) Codec(6,AAC,Passthrough) BSF(AAC_ADTS) Samplerate(48.0K) Format(fltp, 32) Channel(stereo, 2) timebase(1/90000)
	Audio Track #3: Public Name(Audio_257) Variant Name(opus_audio) Bitrate(128.00Kb) Codec(8,OPUS,default) BSF(OPUS) Samplerate(48.0K) Format(s16, 16) Channel(stereo, 2) timebase(1/48000)

Could it be that the data track did not come from OBS where I ran the test stream, but rather from the Image encode I have defined in my OutputProfiles?

<OutputProfiles>


                                                <OutputProfile>
                                                        <Name>stream</Name>
                                                        <OutputStreamName>${OriginStreamName}</OutputStreamName>

                                                        <Encodes>
                                                                <Video>
                                                                        <Name>original_video</Name>
                                                                        <Bypass>true</Bypass>
                                                                </Video>
                                                                <Audio>
                                                                        <Name>aac_audio</Name>
                                                                        <Codec>aac</Codec>
                                                                        <Bitrate>128000</Bitrate>
                                                                        <Samplerate>48000</Samplerate>
                                                                        <BypassIfMatch>
                                                                                <Codec>eq</Codec>
                                                                        </BypassIfMatch>
                                                                </Audio>
                                                                <Audio>
                                                                        <Name>opus_audio</Name>
                                                                        <Codec>opus</Codec>
                                                                        <Bitrate>128000</Bitrate>
                                                                        <Samplerate>48000</Samplerate>
                                                                        <BypassIfMatch>
                                                                                <Codec>eq</Codec>
                                                                        </BypassIfMatch>
                                                                </Audio>
                                                        </Encodes>
                                                </OutputProfile>

                                                <OutputProfile>
                                                        <Name>thumbnail</Name>
                                                        <OutputStreamName>${OriginStreamName}_preview</OutputStreamName>
                                                        <Encodes>
                                                                <Image>
                                                                        <Codec>jpeg</Codec>
                                                                        <Framerate>1</Framerate>
                                                                        <Width>224</Width>
                                                                        <Height>128</Height>
                                                                </Image>
                                                        </Encodes>
                                                </OutputProfile>

                                        </OutputProfiles>

[bchah]

Quick followup - I might be wrong about this clue since the data track also seems to appear when I stream in with RTMP, but then client playback works OK. This issue also occurred when streaming with WHIP. All from OBS with these settings (Except AAC for RTMP):

Here is a debug log if it helps:

[2025-01-06 03:18:29.524] I [OutboundWorker:1642] MediaRouter | mediarouter_application.cpp:481  | [#default#live/helloWorld(2538212941)] Stream has been prepared 
[Stream Info]
id(2538212941), msid(0), output(helloWorld), SourceType(Transcoder), RepresentationType(Source), Created Time (Mon Jan  6 03:18:29 2025) UUID(955ed55d-00ed-405a-87f2-a1cb66223c2c/default/#default#live/helloWorld/o)
	>> Origin Stream Info
	id(100), output(helloWorld), SourceType(WebRTC), Created Time (Mon Jan  6 03:18:26 2025)

	Data  Track #0: Public Name(Data_1) Variant Name(Data) Codec(0,Unknown,Passthrough) BSF(Unknown) timebase(1/1000)
	Audio Track #1: Public Name(Audio_1883117574) Variant Name(aac_audio) Bitrate(128.00Kb) Codec(6,AAC,default) BSF(AAC_ADTS) Samplerate(48.0K) Format(s16, 16) Channel(stereo, 2) timebase(1/48000)
	Audio Track #2: Public Name(Audio_1883117574) Variant Name(opus_audio) Bitrate(128.00Kb) Codec(8,OPUS,Passthrough) BSF(OPUS_RTP_RFC_7587) Samplerate(48.0K) Format(fltp, 32) Channel(stereo, 2) timebase(1/48000)
	Video Track #3: Public Name(Video_1883117575) Variant Name(original_video) Bitrate(1.47Mb) Codec(1,H264,Passthrough:0) BSF(H264_RTP_RFC_6184) Resolution(960x540) Framerate(30.86) KeyInterval(25/frame) SkipFrames(-1) BFrames(0) timebase(1/90000)
[2025-01-06 03:18:29.525] I [OutboundWorker:1642] WebRTC Publisher | rtc_stream.cpp:163  | RtcStream(#default#live/helloWorld) - Ignore unsupported codec(AAC)
[2025-01-06 03:18:29.525] I [OutboundWorker:1642] WebRTC Publisher | rtc_stream.cpp:224  | WebRTC Stream has been created : helloWorld/2538212941
Rtx(false) Ulpfec(false) JitterBuffer(true) PlayoutDelay(false min:0 max: 0)
[2025-01-06 03:18:29.525] I [OutboundWorker:1642] Publisher | stream.cpp:212  | WebRTC Publisher Application has started [helloWorld(2538212941)] stream (MSID : 0)
[2025-01-06 03:18:29.526] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:1108 | LLHlsStream::ComputeOptimalPartDuration() - Audio track(1) SampleRate(48000) frame_duration_ms(21.333333) optimal_frame_count(23.000000) part_target(500.000000) optimal_part_target(490.666667)
[2025-01-06 03:18:29.526] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:1133 | LLHlsStream::AddPackager() - Track(1) ChunkDuration(491.000000)
[2025-01-06 03:18:29.526] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:127  | LLHlsStream(#default#live/helloWorld) - Ignore unsupported codec(OPUS)
[2025-01-06 03:18:29.526] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:1120 | LLHlsStream::ComputeOptimalPartDuration() - Video track(3) FrameRate(30.858245) frame_duration_ms(32.406250) optimal_frame_count(15.000000) part_target(500.000000) optimal_part_target(486.093750)
[2025-01-06 03:18:29.527] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:1133 | LLHlsStream::AddPackager() - Track(3) ChunkDuration(486.000000)
[2025-01-06 03:18:29.527] I [OutboundWorker:1642] LLHLS Publisher | llhls_stream.cpp:239  | LLHlsStream has been created : helloWorld/2538212941
OriginMode(false) Chunk Duration(0.50) Segment Duration(4.00) Segment Count(5) DRM(none)
[2025-01-06 03:18:29.527] I [OutboundWorker:1642] Publisher | stream.cpp:212  | LLHLS Publisher Application has started [helloWorld(2538212941)] stream (MSID : 0)
[2025-01-06 03:18:29.527] I [OutboundWorker:1642] Publisher | stream.cpp:212  | FilePublisher Application has started [helloWorld(2538212941)] stream (MSID : 0)
[2025-01-06 03:18:29.527] I [OutboundWorker:1642] Publisher | stream.cpp:212  | PushPublisher Application has started [helloWorld(2538212941)] stream (MSID : 0)
[2025-01-06 03:18:29.528] W [OutboundWorker:1642] Thumbnail | thumbnail_stream.cpp:49   | Stream [#default#live/helloWorld] was not created because there were no supported codecs by the Thumbnail Publisher.
[2025-01-06 03:18:29.528] W [OutboundWorker:1642] Publisher | application.cpp:336  | ThumbnailPublisher Application could not start [helloWorld] stream.
[2025-01-06 03:18:29.528] I [OutboundWorker:1642] SRTPublisher | srt_stream.cpp:105  | [#default#live/helloWorld(2538212941)] SrtStream - Ignore unsupported codec(OPUS)
[2025-01-06 03:18:29.528] E [OutboundWorker:1642] MPEG-2 TS | mpegts_packetizer.cpp:64   | Data Track (Unknown bitstream format) is not supported
[2025-01-06 03:18:29.528] E [OutboundWorker:1642] SRTPublisher | srt_stream.cpp:124  | [#default#live/helloWorld(2538212941)] Failed to add track to packetizer
[2025-01-06 03:18:29.528] W [OutboundWorker:1642] Publisher | application.cpp:336  | SRTPublisher Application could not start [helloWorld] stream.
[2025-01-06 03:18:30.464] W [OutboundWorker:1642] MediaRouter | mediarouter_stream.cpp:1227 | [#default#live/helloWorld] Detected out of order DTS of packet. track_id:2 dts:146402->146375
[2025-01-06 03:18:31.269] I [InboundWorker:1727] Transcoder | decoder_avc.cpp:248  | [#default#live/helloWorld(100)] input track information: [video] h264 (Main 3.1), yuvj420p, 960x540 [SAR 1:1 DAR 16:9], 30 fps, 195 kbps, timebase: 1/60, 
[2025-01-06 03:18:31.270] I [InboundWorker:1727] Transcoder | transcoder_encoder.cpp:283  | The encoder has been created. track(#5), codec(JPEG), module(default:0)
[2025-01-06 03:18:31.271] I [FLT-rscl-t5:1758] Transcoder | filter_rescaler.cpp:331  | Rescaler parameters. track(#1883117575 -> #5), module(default:0 -> default:0). desc(src:video_size=960x540:pix_fmt=yuvj420p:time_base=1/90000:pixel_aspect=1/1 -> output:settb=1/90000,scale=224x128:flags=bilinear,format=yuvj420p), fps(30.17 -> 1.00), skipFrames(0)
[2025-01-06 03:18:31.272] W [FLT-rscl-t5:1758] FFmpeg | third_parties.cpp:115  | [SWScaler: 0x7f0bc1e56b40] deprecated pixel format used, make sure you did set range correctly
[2025-01-06 03:18:31.273] W [FLT-rscl-t5:1758] FFmpeg | third_parties.cpp:115  | [SWScaler: 0x7f0bc1efdd80] deprecated pixel format used, make sure you did set range correctly
[2025-01-06 03:18:31.273] W [FLT-rscl-t5:1758] FFmpeg | third_parties.cpp:115  | [SWScaler: 0x7f0bc1f2b300] deprecated pixel format used, make sure you did set range correctly
[2025-01-06 03:18:31.273] W [FLT-rscl-t5:1758] FFmpeg | third_parties.cpp:115  | [SWScaler: 0x7f0bc1f5bc00] deprecated pixel format used, make sure you did set range correctly
[2025-01-06 03:18:31.473] I [SPRtcSig-t3333:1602] Signalling | rtc_signalling_server.cpp:316  | New client is connected: 
[2025-01-06 03:18:31.593] I [SPRtcSig-t3333:1602] Monitor | stream_metrics.cpp:114  | A new session has started playing #default#live/helloWorld on the WebRTC publisher. WebRTC(1)/Stream total(1)/App total(1)
[2025-01-06 03:18:31.593] I [SPRtcSig-t3333:1602] ICE | ice_port.cpp:368  | Added session: 1 (ufrag: aAHEZu:s/Im)
[2025-01-06 03:18:31.666] I [SPICE-t10010:1612] ICE | ice_port.cpp:606  | Turn client has connected : 
[2025-01-06 03:18:31.681] I [SPICE-t10010:1612] ICE | ice_port.cpp:843  | Session 1 uses candidate: 
[2025-01-06 03:18:33.457] I [AW-LLHLS0:1649] LLHLS Publisher | llhls_stream.cpp:1368 | LLHlsStream(#default#live/helloWorld) - Ready to play : Part Hold Back = 2.000000
[2025-01-06 03:18:38.384] W [AW-LLHLS0:1649] FMP4 Packager | fmp4_storage.cpp:347  | LLHLS stream (#default#live/helloWorld) / track (3) - a longer-than-expected (5412.0 | expected : 4000) segment has created. Long or irregular keyframe interval could be the cause.
[2025-01-06 03:18:40.657] W [OutboundWorker:1642] MediaRouter | mediarouter_stream.cpp:1227 | [#default#live/helloWorld] Detected out of order DTS of packet. track_id:2 dts:635701->635697
[2025-01-06 03:18:42.328] I [SRT:RcvQ:w1:1626] SRT | queue.cpp@SRT:1413 | request from: 172.17.0.1:60311 to agent:564988031
[2025-01-06 03:18:42.328] I [SRT:RcvQ:w1:1626] SRT | queue.cpp@SRT:1432 | managed the connection request from: 172.17.0.1:60311 result:waveahand
[2025-01-06 03:18:42.335] I [SRT:RcvQ:w1:1626] SRT | queue.cpp@SRT:1413 | request from: 172.17.0.1:60311 to agent:564988031
[2025-01-06 03:18:42.336] I [SRT:RcvQ:w1:1626] SRT | core.cpp@SRT:2069 | cmd=1(HSREQ) len=12 vers=0x10404 opts=0xbf delay=20
[2025-01-06 03:18:42.336] I [SRT:RcvQ:w1:1626] SRT | core.cpp@SRT:11117 | @564988031: listen ret: -1 - conclusion
[2025-01-06 03:18:42.337] I [SRT:RcvQ:w1:1626] SRT | queue.cpp@SRT:1432 | managed the connection request from: 172.17.0.1:60311 result:waveahand
[2025-01-06 03:18:42.337] I [SPSRTPub-s9998:1624] SRTPublisher | srt_publisher.cpp:318  | The SRT client has connected: 
[2025-01-06 03:18:42.338] I [SPSRTPub-s9998:1624] Monitor | stream_metrics.cpp:114  | A new session has started playing #default#live/helloWorld on the SRT publisher. SRT(1)/Stream total(2)/App total(2)
[2025-01-06 03:18:45.762] W [OutboundWorker:1642] MediaRouter | mediarouter_stream.cpp:1227 | [#default#live/helloWorld] Detected out of order DTS of packet. track_id:2 dts:880884->880702
[2025-01-06 03:18:48.386] I [SPICE-t10010:1612] ICE | ice_port.cpp:620  | Turn client has disconnected : 
[2025-01-06 03:18:48.386] I [SPRtcSig-t3333:1602] WebRTC Publisher | webrtc_publisher.cpp:534  | Stop command received : #default#live/helloWorld/100
[2025-01-06 03:18:48.387] W [StreamWorker:1733] Socket | socket.cpp:1058 | [#44] [0x7f0c17bfde10] IsSendable(): Invalid state: Disconnected (expected: == SocketState::Connected) - 
[2025-01-06 03:18:48.387] W [StreamWorker:1733] Socket | socket.cpp:1058 | [#44] [0x7f0c17bfde10] IsSendable(): Invalid state: Disconnected (expected: == SocketState::Connected) - 
[2025-01-06 03:18:48.388] W [SPRtcSig-t3333:1602] Socket | socket.cpp:1058 | [#44] [0x7f0c17bfde10] IsSendable(): Invalid state: Disconnected (expected: == SocketState::Connected) - 
[2025-01-06 03:18:48.388] I [SPRtcSig-t3333:1602] Monitor | stream_metrics.cpp:136  | A session has been stopped playing #default#live/helloWorld on the WebRTC publisher. Concurrent Viewers[WebRTC(0)/Stream total(1)/App total(1)]
[2025-01-06 03:18:48.388] I [SPRtcSig-t3333:1602] ICE | ice_port.cpp:436  | Removed session(1) from ICEPort | ice_seesions_with_id count(1) ice_sessions_with_ufrag(1) ice_sessions_with_address_pair(1) 
[2025-01-06 03:18:48.388] I [SPRtcSig-t3333:1602] Signalling | rtc_signalling_server.cpp:450  | Client is disconnected: HttpConnection(0x7f0c17b21610) : WebSocket  TLS(Disabled) (#default#live / helloWorld, ufrag: local: aAHEZu, remote: s/Im)

But maybe this is already fixed with your latest commit, which I will test as soon as possible! Thank you again 🥇

[dimiden]

@bchah
Thank you for the information. I will look into this. 👍

[bchah]

It's working great now! With H265 also. Great work and thank you again ⭐️

[bchah]

@dimiden I notice that the SRT Publisher is not included in the Stream Statistics API. Please consider adding it soon, thank you 👍