Getting HTTP based playback to work?

[NattyNarwhal]

Hi, I'm still getting my Objective-C legs still, so pardon me if I sound a bit dumb.

I'm adapting an application from the before times of Snow Leopard to Monterey. There's a lot to do still (i.e. convert to ARC, clean up UI threading issues, etc.), but I figured getting working audio would be a nice dopamine rush. It used both QTKit and SFBAudioEngine for playback, depending if the file was remote or local, respectively. Since QTKit has been mega deprecated, and porting it to the modern AVFramework seems like a bit of a feat, I figured, since SFBAudioEngine supports both local and remote playback, why not just use SFBAudioEngine all the time?

I did have to do the conversion from the C++ to the Objective-C API, which I'm still figuring out (i.e. going to do callback to delegate conversion, the measurement type changes), but the biggest thing is using SFBAudioEngine to play audio from an HTTP service.

Mildly cleaned up example:

- (void)playWithURL:(NSURL *)url {
    NSError *decodeError = nil;
    // Manually specifying a decoder because MIME type detection is wonky. If not set, then we bomb out here.
    // CoreAudio decoder also has trouble.
    SFBAudioDecoder *decoder = [[SFBAudioDecoder alloc] initWithURL: url decoderName:SFBAudioDecoderNameMPEG error: &decodeError];
    if (NULL != decoder) {

        [localPlayer setVolume: [self volume] error: nil];

        // Register for rendering started/finished notifications so the UI can be updated properly
        // <callbacks are set here>
        NSError *decoderError = nil;
        [decoder openReturningError: &decoderError];
        if (decoderError) {
            // Where we usually end up.
            NSLog(@"Decoder open error: %@", decoderError);
            [decoder dealloc];
            return;
        }
        if ([localPlayer enqueueDecoder: decoder error: nil]) {
            // if it'd get here, it'd work
        }else {
            [decoder dealloc];
        }
    } else {
        NSLog(@"Couldn't decode %@: %@", url, decodeError);
    }
}

Here's the full console log output from Xcode. Apologies for having other application issues intermixed; I'll clean it up, but don't know how much is relevant to my issue with SFBAudioEngine specifically. I do notice in particular that the NSURLSession downloader stuff seems to be triggering multiple times and gets cancelled. I wonder if that has something to do with it?

If I curl the URL, I get a perfectly normal file, without any special headers or whatever, so I don't think it's the HTTP input source choking on not having proper auth or anything.

In addition, I've had difficulty with the MIME type detection (since extension won't work for the API endpoints) not well - not sure if it's enumerating through all the different providers. I do know one type was missing (audio/mp4).

[sbooth]

The HTTP input source pathway is the least-tested so it's definitely possible SFBAudioEngine is doing something incorrectly with NSURLSessionDataTask, especially when it comes to seeking.

If you try to play the URL using SimplePlayer do you see the same behavior? Or does opening fail completely due to lack of MIME type?

Interestingly, audio/mp4 is not a MIME type returned by Core Audio (MP4 decoding in SFBAudioEngine is handled by Core Audio).

[NattyNarwhal]

For another reply (if GitHub works today...): I tried using a normal Apache server (instead of the web application), serving the file as an .mp3 file (so it guesses based on extension fine if that matters) - still the same error, though TagLib is also in the mix now?

2022-03-17 12:12:38.023863-0300 SimplePlayer[54430:4109433] [AudioPlayerNode] Render block format: <AVAudioFormat 0x6000016c5f90:  2 ch,  44100 Hz, Float32, non-inter>
2022-03-17 12:12:38.023962-0300 SimplePlayer[54430:4109777] [AudioPlayerNode] Decoder thread starting
2022-03-17 12:12:38.023972-0300 SimplePlayer[54430:4109778] [AudioPlayerNode] Notifier thread starting
2022-03-17 12:12:38.026570-0300 SimplePlayer[54430:4109433] [plugin] AddInstanceForFactory: No factory registered for id <CFUUID 0x60000358d460> F8BB1C28-BAE8-11D6-9C31-00039315CD46
2022-03-17 12:12:38.039315-0300 SimplePlayer[54430:4109433]  HALC_ProxyObjectMap::_CopyObjectByObjectID: failed to create the local object
2022-03-17 12:12:38.039366-0300 SimplePlayer[54430:4109433]  HALC_ShellDevice::RebuildControlList: couldn't find the control object
2022-03-17 12:12:38.102414-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102776-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102821-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102862-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102896-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102943-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.102979-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.103012-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.103043-0300 SimplePlayer[54430:4109433] throwing -10878
2022-03-17 12:12:38.108002-0300 SimplePlayer[54430:4109433] [AudioPlayer] AVAudioEngine output format changed from <AVAudioFormat 0x6000016fdf40:  2 ch,  44100 Hz, Float32, non-inter> to <AVAudioFormat 0x6000016f8a50:  2 ch,  48000 Hz, Float32, non-inter>
2022-03-17 12:12:38.109279-0300 SimplePlayer[54430:4109433] [AudioPlayer] ↑ rendering: <AVAudioFormat 0x6000016c5f90:  2 ch,  44100 Hz, Float32, non-inter>
2022-03-17 12:12:38.109310-0300 SimplePlayer[54430:4109433] [AudioPlayer] ← main mixer out: <AVAudioFormat 0x6000016f8c80:  2 ch,  48000 Hz, Float32, non-inter>
2022-03-17 12:12:38.181201-0300 SimplePlayer[54430:4109433] [AudioPlayer] Setting output device to 0x38
2022-03-17 12:12:38.253600-0300 SimplePlayer[54430:4109433] Metal API Validation Enabled
TagLib: Could not open file /test.mp3
2022-03-17 12:12:56.473934-0300 SimplePlayer[54430:4109775] [InputSource] NSURLSessionData error: Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=http://192.168.2.93/test.mp3, NSErrorFailingURLKey=http://192.168.2.93/test.mp3, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <57D1E151-8B37-474E-8D2D-F6D0DE3B6A37>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <57D1E151-8B37-474E-8D2D-F6D0DE3B6A37>.<1>, NSLocalizedDescription=cancelled}
2022-03-17 12:12:56.473983-0300 SimplePlayer[54430:4109801] [InputSource] NSURLSessionData error: Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=http://192.168.2.93/test.mp3, NSErrorFailingURLKey=http://192.168.2.93/test.mp3, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <2A4A3A41-CFAA-41A0-9556-E448F0B86307>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <2A4A3A41-CFAA-41A0-9556-E448F0B86307>.<1>, NSLocalizedDescription=cancelled}
2022-03-17 12:12:56.559988-0300 SimplePlayer[54430:4109801] [InputSource] HTTP status: 200 no error

[sbooth]

TagLib is involved when adding an item to the playlist because an attempt is made to read properties and metadata. That error message is unrelated to playback and can be safely ignored.

I can reproduce the problem you're experiencing with MP3 playback via HTTP. While FLAC works fine, MP3 seems to be failing around line 174 of SFBMPEGDecoder.m.

[NattyNarwhal]

I wasn't able to get FLAC working with the web application's HTTP server, but I haven't tried it in Apache yet.

FWIW, looking in Wireshark, it does seem it slurps the entire file at once instead of doing any seeking at the HTTP level, or if it is, it's made pointless by requesting the whole thing.

[NattyNarwhal]

And yeah, I had stepped into mpg123_getformat; it seemingly eventually tries to do a read, and I think that's failing?

[sbooth]

After looking at this a bit more I think one of the main causes (in SimplePlayer at least) is that the decoder is opened immediately following the creation of the data task before the request actually has a chance to return any data. This could be the issue you're experiencing as well.

I'm not sure yet what the best fix would be. It probably makes sense to wait for the data task to receive something before starting playback.

One option is to expose SFBHTTPInputSource and add the ability to query the data task status. This would put some burden on the application but might be more flexible in the long run.