Audio Quality is low

[vnapps-feedback]

Hello,

I installed Ant Media Server with Default Settings.
I'm streaming an music file (filetype: mp3, sample rate: 44100, channels: stereo, bitrate: 128kbps) to Ant Media Server using iOS SDK and open the stream on Ant Media Server management web.
I see that the audio quality on web player is really low (not the same as I open the music file directly on my macbook).
How can I improve the audio quality?
Have any settings on Ant Media Server to use stereo audio and bitrate: 128kbps audio?

[lastpeony]

Hi
Please check this github issue answer: #4929 (comment)
In summary:
WebRTC is a network-dependent protocol, and the quality of streams is mainly influenced by network conditions and bandwidth. WebRTC does not allow for forcing quality in publishing, but it dynamically adjusts resolution and bitrate based on available bandwidth.
However, there are settings and tuning options available to optimize WebRTC usage. For example, in the JavaScript SDK, you can modify the maxVideoBitrateKbps variable to adjust the incoming bitrate. It is recommended to set it based on your specific use case and consider the bandwidth limitations of users.
Resolution can also be set through media constraints in the media manager. On the Android SDK, you can specify the maximum bitrate using the EXTRA_VIDEO_BITRATE parameter. Similarly, on the iOS side, you can set the max bitrate and target resolution using appropriate methods.
It is important to note that exact bitrates cannot be set all the time. Instead, a maximum bitrate level can be specified, and the actual bitrate will dynamically adjust based on network conditions.

[yashtandon113]

@vnapps-feedback
We are already working on Stereo Audio following the below issue. We will share the snapshot build with you for the testing asap.
#3076

[mekya]

Hi @vnapps-feedback,
I'm also trying to push a mp3 file using iOS SDK. Could you please make a contribution? So that I can pass this issue and focus on increasing audio quality on the server side.

Regards
Oguz

[vnapps-feedback]

Ok @mekya , I sent mp3 file using AVAudioFile to read to AVAudioPCMBuffer after I convert AVAudioPCMBuffer -> CMSampleBuffer
and using self.publisherClient?.deliverExternalAudio(sampleBuffer: cmSampleBuffer!) to publish audio data.

This is function to convert AVAudioPCMBuffer to CMSampleBuffer

`
public func toCMSampleBuffer(from pcmBuffer: AVAudioPCMBuffer, decodeTimeStamp: CMTime, presentationTimeStamp: CMTime, duration: CMTime) -> CMSampleBuffer? {

 let asbd = pcmBuffer.format.streamDescription
    if asbd.pointee.mSampleRate != 44100
        || asbd.pointee.mFormatID != kAudioFormatLinearPCM
        || asbd.pointee.mFormatFlags != 0xe
        || asbd.pointee.mBytesPerPacket != 4
        || asbd.pointee.mFramesPerPacket != 1
        || asbd.pointee.mBytesPerFrame != 4
        || asbd.pointee.mChannelsPerFrame != 2
        || asbd.pointee.mBitsPerChannel != 16 {
        outASBD = AudioStreamBasicDescription(
                mSampleRate: 44100,
                mFormatID: kAudioFormatLinearPCM,
                mFormatFlags: 0xe,
                mBytesPerPacket: 4,
                mFramesPerPacket: 1,
                mBytesPerFrame: 4,
                mChannelsPerFrame: 2,
                mBitsPerChannel: 16,
                mReserved: 0)
        let outFormat = AVAudioFormat(streamDescription: &(outASBD!))

        if converter == nil{
            converter = AVAudioConverter(from: pcmBuffer.format, to: outFormat!)
        }
        outBuffer = AVAudioPCMBuffer(pcmFormat: outFormat!, frameCapacity: AVAudioFrameCount(pcmBuffer.frameLength))

        let inputBlock : AVAudioConverterInputBlock = {
            inNumPackets, outStatus in
            outStatus.pointee = AVAudioConverterInputStatus.haveData
            return pcmBuffer
        }
        converter!.convert(to: outBuffer!, error: &error, withInputFrom: inputBlock)
    } else {
        outBuffer = pcmBuffer
        outASBD = asbd.pointee
    }
    let audioBufferList = outBuffer!.mutableAudioBufferList

    var sampleBuffer: CMSampleBuffer? = nil
    var format: CMFormatDescription? = nil
    
    var status = CMAudioFormatDescriptionCreate(allocator: kCFAllocatorDefault,
                                                asbd: &(outASBD!),
                                                layoutSize: 0,
                                                layout: nil,
                                                magicCookieSize: 0,
                                                magicCookie: nil,
                                                extensions: nil,
                                                formatDescriptionOut: &format);
    
    if (status != noErr) { return nil; }
    
    var timing: CMSampleTimingInfo = CMSampleTimingInfo(duration: duration,
                                                        presentationTimeStamp: presentationTimeStamp,
                                                        decodeTimeStamp: decodeTimeStamp)
    
    status = CMSampleBufferCreate(allocator: kCFAllocatorDefault,
                                  dataBuffer: nil,
                                  dataReady: false,
                                  makeDataReadyCallback: nil,
                                  refcon: nil,
                                  formatDescription: format,
                                  sampleCount: CMItemCount(outBuffer!.frameLength),
                                  sampleTimingEntryCount: 1,
                                  sampleTimingArray: &timing,
                                  sampleSizeEntryCount: 0,
                                  sampleSizeArray: nil,
                                  sampleBufferOut: &sampleBuffer);
    if (status != noErr) {
        print("CMSampleBufferCreate returned error: \(status)");
        return nil
        
    }
    
    status = CMSampleBufferSetDataBufferFromAudioBufferList(sampleBuffer!,
                                                            blockBufferAllocator: kCFAllocatorDefault,
                                                            blockBufferMemoryAllocator: kCFAllocatorDefault,
                                                            flags: 0,
                                                            bufferList: audioBufferList);
    if (status != noErr) {
        print("CMSampleBufferSetDataBufferFromAudioBufferList returned error: \(status)");
        return nil;
        
    }
    
    return sampleBuffer
}

`
If you need more information let me know.

[mekya]

Thank you @vnapps-feedback. I'll try that.

I have a snapshot that supports stereo and relays the audio as is to WebRTC viewers.

Could you please try it and let me know your experience?
https://drive.google.com/file/d/1f8DfYvc3K1crTjB-W6Vv7RYjSqRs4hhY/view?usp=drive_link

[vnapps-feedback]

Hi mekya,
I tried the new version of
ant media server Enterprise Edition 2.6.0-SNAPSHOT 20230526_0950
I don't see a noticeable improvement in sound quality. The sound quality is still not as good as when I listen to the mp3 file directly. I have attached the project I used to stream mp3 files for your reference.

https://drive.google.com/file/d/1O9wcX0Y5kuIzNTeExPcjbwV55kZ1Ms-r/view?usp=share_link

[mekya]

Hi @vnapps-feedback,
I hope you have not changed any settings in the snapshot that I've shared. Because I've shared with one of the user(friend), he did put it into his cluster and didn't see any improvement because cluster settings are overwritten to default settings I've provided.
After he run the server in standalone mode, he confirmed that it's streaming with high frequency and it's suitable for music streaming.

If you have a similar issue, you can check it on your end.
If you still have a problem, then iOS SDK may decrease the quality of the stream. I try to check your mp3 files and update you.

PS: I just have a chance to try your code and I'll let you know the result. Thank you for your contribution.

[mekya]

I have an idea.

Could you please publish your screen(broadcast extension) in iOS SDK and go to youtube in your iOS device and open a music video and check whether it's playing well or not through Ant Media Server?

[vnapps-feedback]

Hi @mekya,
I tried as your suggestion. The quality is not the same. Please see this video. I recorded screen broadcast from my iPhone and soud from playing that song directly in Web. The quality of playing directly song in web is more better. https://drive.google.com/file/d/1fGfROv8BkTJBIRJnBvI6HF3lS5uSmTmy/view?usp=sharing

Could you share me your server for testing. I don't know if my server is configured correctly.

[vnapps-feedback]

Please compare sound at 00:21 and 1:04 (2 version of same position in the song)

[mekya]

I see the video. Then the problem with the MP3 stuff is that the audio should be sent every 20ms or 10ms of CMSampleBuffer.
Giving the whole CMSampleBuffer to the WebRTC stack creates a quality problem you encounter.

The primary point is on the iOS side not on the server side. Secondly, in order to play high-frequency audio on the server side, the recent version will help.

If it's ok for you, let's proceed with the iOS side mp3 first and in that time I'll try to provide some test servers to you.

Btw, I'm still not sending 20ms or 10ms of CMSampleBuffer by dividing big CMSamplebuffer into smaller parts. Do you have some code for this one?

[vnapps-feedback]

Hi @mekya ,
I don't think the reason is from whole CMSampleBuffer

I splited CMSampleBuffer to only 255 frames/package. But the quality is the same (quite low).
I using ScreenSharing for testing the quality still low
I using Android for publishing, the music quality still low
=> I think the problem is from Server side.
Do you have any suggestion? Could you give me a server for testing?

[vnapps-feedback]

Hi @mekya,
Do you have any new information? I’m waiting for a suggestion to solve audio quality issue.

[mekya]

Sorry for not being responsive. I've focused on several things.

If you use ScreenSharing and quality is still low, then it's about sample rate and our latest snapshot resolves this problem. It supports streaming 48Khz sample rate, in older versions it decreases the sample rate to 16Khz and it decreases the audio quality.

Let's schedule a meeting next week to discuss and finalize the things. How's your availability in next week?

Btw, I've having difficulty to split the CMSampleBuffer. I'll try to take a look again. If you have something for me, please share it with me

[vnapps-feedback]

Hi @mekya,
I upgraded my server to the lastest snapshot v2.6.3 but the quality hasn't still improved.
I wonder if the upgrade is ok with this command (sudo ./install_ant-media-server.sh -i ant-media-server-enterprise-2.6.3.zip -r true)
I'm available all days in week. Please contact me when you're available..

About CMSampleBuffer, I don't really Split it. I create AVAudioPCMBuffer with small package (about 255 Frames/package), after that I convert AVAudioPCMBuffer to CMSampleBuffer, therefore CMSampleBuffer only contains about 255 frames/package also.

[mekya]

Hi @vnapps-feedback,

2.6.3-SNAPSHOT is not direct solution we've a separated SNAPSHOT that has a support for high sample rate(48Khz).

Let's schedule some time to make the test together because it will save good amount of time for both of us at least to understand the problem clearly. If it's ok for you, let me know your availability for this week and next week.

Cheers

[vnapps-feedback]

Hi @mekya ,
I'm available all working time (8AM-11AM, 14PM-17PM Indochina time zone (UTC+7) from Monday to Friday this week and next week). Feel free to contact me any time. Thank you. I'm waiting for testing with you.

[mojatter]

Hi @vnapps-feedback and @mekya

This is for the enterprise version. if you specify settings.audioBitrateSFU in red5-web.properties, the sound quality will improve.

settings.audioBitrateSFU=256000

[mekya]

hi @mojatter,
Thank you for your contribution. Appreciate that. Yes, it improves the audio quality bitrate. On the other hand, we also need to improve the sample rate. Why don't you join the meeting with @vnapps-feedback ?

[vnapps-feedback]

Hi @mojatter ,
Thank you for suggestion. I tried setting audioBitrateSFU but the audio quality is still low. Hi @mekya, please let me know if you're available. I'm waiting for test your snapshot with Sample rate of 48 kHz. It's really potential.

[mekya]

Hi @vnapps-feedback,

Let's make it tomorrow at 3.30pm UTC+7 and it's 11.30 am UTC+3.

If you're availble @mojatter, please join the meeting.

Meeting with Đặng Tuấn Anh about Audio Quality
Thursday, July 6 · 11:30am – 12:00pm
Time zone: Europe/Istanbul
Google Meet joining info
Video call link: https://meet.google.com/ebr-qauv-kwn

[vnapps-feedback]

Hi @mekya,
Ok bro.
See you tomorrow.

[mekya]

See you soon bro :)

[vnapps-feedback]

Hi @mekya,
I think the quality of this server version is better. Because I harder to hear the difference in audio quality in the last time.
Please send me the snapshot. I need more time to test and I will give you the feedback soon.
Thank you for your time.

[vnapps-feedback]

Hi @mekya, could you share me the snapshot with 48000Hz supported. I want to test more. Thank you.

[mekya]

Hi @vnapps-feedback,
It's my bad. I could not answer earlier to this email due to load on my end. Let me prepare the things and share it with you.

[mekya]

You can still use this file https://drive.google.com/file/d/1f8DfYvc3K1crTjB-W6Vv7RYjSqRs4hhY/view?usp=drive_link

I think you need to check high freq to understand there is a problem or not.

Here is a file includes high frequency audio.
https://drive.google.com/file/d/1tqkHKWqA5uZk6hvH0usWu21P2R6XrpmC/view?usp=sharing

Just replace this file with adele or michale jackson files in the /usr/local/antmedia/webapps/WebRTCAppEE folder or edit the
/usr/local/antmedia/webapps/WebRTCAppEE/stereo_publish.html to use this mp3 file