Make decodeAudioData more precise

[annevk]

It would be nice if https://webaudio.github.io/web-audio-api/#dom-baseaudiocontext-decodeaudiodata invoked an algorithm in MIME Sniffing rather than just referencing the whole standard. It would also be good to say which return values are accepted/rejected.

Are there tests for this method?

[padenot]

We're in the process of upstreaming tests from the various UAs into web-platform-tests, I don't now where we're at right now, but there certainly will be tests.

[rtoy]

Chrome has basically skipped upstreaming the decoder tests. I think the only thing that might really be cross-platform would be decoding 16-bit PCM WAV files. We could probably move the couple of tests that we have for this.

[rtoy]

Do you mean we should probably reference https://mimesniff.spec.whatwg.org/#audio-or-video-type-pattern-matching-algorithm ?

[annevk]

Yeah, I guess. Whatever matches implementations. (That's also why I'm interested in tests, I wonder if this is specified properly.)

[rtoy]

Chrome lets ffmpeg do all the sniffing. I don't know how that relates to the mimesniff algorithm, but it must be somewhat similar.

Chrome has no explicit tests on sniffing; we only check that the decoded file matches what we thought it should be.

[agnivade]

@rtoy - Does that mean a web server cannot reliably sniff audio content without calling into ffmpeg ?

Because without any algorithm it is not possible to reliably do mime sniffing.

Context:
I would like to implement mime sniffing for mp3s in Go's http server. But that is blocked on whatwg/mimesniff#70. Which is blocked on this issue. Would appreciate any leads on this.

[rtoy]

@agnivade: Don't have anything really to add. Mime sniffing is just a heuristic anyway, right?

[agnivade]

Sure, that maybe. But there are major differences between the mime sniffing algorithm in https://mimesniff.spec.whatwg.org/#signature-for-mp3-without-id3 and the actual algorithm used by mozilla
https://dxr.mozilla.org/mozilla-central/source/toolkit/components/mediasniffer/mp3sniff.c.

We should fix that. I tried to implement what was in mimesniff and it failed for every possible mp3 out there. I don't believe that should be the case.

[annevk]

It's not exactly a heuristic. It determines whether or not a byte stream can be played back and needs to be interoperable across implementations.

[GPHemsley]

I'm not sure that #1781 sufficiently addressed this.

For starters, it's pointing to the wrong mimesniff algorithm. The rules for sniffing audio and video specifically are what is intended to be used for sniffing in an audio or video context (such as <audio> or <video>). This is distinct from the algorithm identifying audio or video in a browsing context, which is meant to be much more restrictive and geared towards balancing security and legacy content.

Secondly, the spec for decodeAudioData() still contains:

The buffer passed to decodeAudioData() has its content-type determined by sniffing, as described in [mimesniff].

Which I believe is what @annevk was referring to as vague.

[padenot]

I'm not sure that #1781 sufficiently addressed this.

I agree, it's not enough. I've filed #2378 as a follow-up and will address the issues you describe there. Thanks for reporting!