I can't hear audio

[uraltelekomsemihataman]

I have created a simple project that is showing rtsp stream in React according to your documentation and I can display the video but I can't hear the video. I am calling onAudioDecode callback when creating player but this callback is never called.

[uraltelekomsemihataman]

This is my app.js in backend:

const express = require('express');
const app = express();
require('dotenv').config();

const { proxy, scriptUrl } = require('rtsp-relay')(app);

const handler = proxy({
    url: `rtsp://${process.env.RTSP_USERNAME}:${process.env.RTSP_PASSWORD}@${process.env.RTSP_URL}`,
    // if your RTSP stream need credentials, include them in the URL as above
    verbose: false,
    additionalFlags: ['-q', '1'] 
});

// the endpoint our RTSP uses
app.ws('/api/stream', handler);

// this is an example html page to view the stream
app.get('/', (req, res) =>
    res.send(`
  <canvas id='canvas'></canvas>

  <script src='${scriptUrl}'></script>
  <script>
    loadPlayer({
      url: 'ws://' + location.host + '/api/stream',
      canvas: document.getElementById('canvas')
    });
  </script>
`),
);

app.listen(2000);

and this is my react component code:

import { useRef, useState } from "react";
import { loadPlayer } from "rtsp-relay/browser";

export default function RTSPPlayer() {
   const canvasContainerStyle = {
      width: "100%",
      height: "100vh",
      position: "relative",
      left: "50%"
   };

   const canvasStyle = {
      width: "100%",
      height: "90%",
   };

   const buttonsContainerInitialStyle = {
      position: "absolute",
      left: "50%",
      top: "50%"
   };

   const buttonsContainerStyle = {
      position: "relative",
      left: "70%",
      bottom: "3rem"
   };

   const [player, setPlayer] = useState(null);
   const [isPaused, setPaused] = useState(false);
   const canvas = useRef(null);

   const handleStartStreamClick = async () => {
      if (!canvas.current) throw new Error("Ref is null");

      const Player = await loadPlayer({
         url: "ws://localhost:2000/api/stream",
         canvas: canvas.current,
         maxAudioLag: 1,
         audioBufferSize: 1024*1024
      });

      console.log(Player);
      // Player.options.audioOut.unlock(function(){
      //    alert('unlocked!');
      // });

      setPlayer(Player);
   };

   const handlePauseStreamClick = () => {
      player.pause();
      setPaused(true);
   };

   const handleResumeStreamClick = () => {
      player.play();
      setPaused(false);
      console.log(player);
   };


   return (
      <>
         <div style={canvasContainerStyle}>
            <canvas ref={canvas} style={canvasStyle}></canvas>
         </div>
         <div style={player ? buttonsContainerStyle : buttonsContainerInitialStyle}>
            <button onClick={handleStartStreamClick} disabled={player}>Start Stream</button>
            <button onClick={handlePauseStreamClick} disabled={!player || isPaused}>Pause Stream</button>
            <button onClick={handleResumeStreamClick} disabled={!player || !isPaused}>Resume Stream</button>
         </div>
      </>
   );
}

[fullmooooon]

https://github.com/phoboslab/jsmpeg

The source code of rtsp-relay did not specify the audio format as MP2 as required by jsmpeg. After I made this modification, my Electron program played the audio from the RTSP stream.

additionalFlags: `-codec:a mp2 -ar 44100 -ac 1 -b:a 128k`.split(` `),

    const express = require('express')
    const app = express()
    const { proxy, scriptUrl } = require('rtsp-relay')(app)
    const handler1 = proxy({
      url: videoUrl.value,
      transport: 'tcp',
      additionalFlags: `-codec:a mp2 -ar 44100 -ac 1 -b:a 128k`.split(` `),
    })
    app.ws('/api/stream1', handler1)
    app.listen(2001)