Publishing metadata (ideally JSON) to mediamtx endpoint along with RTSP stream possible

[joshting]

Question

Hi all, I am interested in publishing metadata alongside RTSP stream so that when streaming from the web (which I believe MEDIAMTX is using WebRTC protocol), that metadata can be streamed live and in sync (as WebRTC supports both video and text). I am aware this is not standard RTSP but all suggestions are welcomed. I am using Gstreamer and the syncing of the metadata with the video frames are crucial in the work I am doing.

[Edit]
I don't know enough of WebRTC, Pion library and Golang in general to navigate the code and make changes quickly. Hope someone can point me in the right direction to add the ability to add the ability to publish json data to an endpoint and in turn stream it through WebRTC Data Channels.

[aler9]

Hello, this is being developed and is being tracked in #2693, but in the meanwhile nothing prevents you from routing video streams with MediaMTX, routing metadata streams with Kafka or MQTT and then merge the two streams together at client side. I do this all the time.

If you use the RTSP protocol, you also have access to the original timestamp of frames, which you can use for precise syncing.

If you use WebRTC, you should be able to do that too through a fairly recent API upgrade, take a look here:

https://stackoverflow.com/questions/48927871/accessing-rtp-timestamps-from-webrtc/75724300#75724300