Discuss how to reconcile colorGamut & transferFunction with ISO 23001-8:2016 #152

vi-dot-cpp · 2020-03-31T20:39:43Z

Hi everyone. chcunningham@ and I would like to (re)open up discussion with respect to how MediaCapabilities.decodingInfo() should handle discrepancies between colorGamut & transferFunction input and and the color information implicit in mime types like VP9 and AV1.

Take the following example:

navigator.mediaCapabilities.decodingInfo({
    type: 'file',
    video: {
      contentType: 'video/webm; codecs="vp09.02.10.10.01.09.16.09.01"',
      ...
      colorGamut: "srgb",
      transferFunction: "srgb",
    },
  })

The mime type implies BT2020 primaries and SMPTEST2084 EOTF, but colorGamut and transferFunction are specified as BT709. How should UA validate this input? Thanks everyone -- best wishes to everyone.

The text was updated successfully, but these errors were encountered:

tidoust · 2020-05-15T08:16:02Z

Discussed during the Media WG call on 12 May 2020.

Resolution: reject the promise when there are conflicting inputs, details can be discussed back in the issue (see minutes)

vi-dot-cpp · 2020-05-18T14:59:53Z

Thanks tidoust@. jernoble@ rightly pointed out that this reconciliation between properties and mime types should occur broadly. For HDR properties though unique challenges may exist due to how colorGamut and transferFunction are defined.

For instance, chcunningham@ noted that mime types can specify an obscure transfer function not defined by the MediaCapabilities TransferFunction enum.

chcunningham · 2020-06-04T19:33:25Z

@jernoble, thoughts on the last comment from Vi?

jernoble · 2020-06-04T20:15:16Z

I don't think that it changes the outcome. If a MIME type specifies an obscure transfer function, the UA must be able to say "supports: false/true" using only that MIME type already, as the transferFunction property is optional. And if the client specifies a non-matching transferFunction, that should have the same outcome:

UAs which don't support or understand the MIME-specified transfer function will say "supports:false".
UAs which do will see that there's a conflict between the MIME-specified transfer function and the client specified transferFunction parameter and also say "supports:false".

So lets look at the forward compatibility story:

If we add new values to the transferFunction enum, how will clients know what values are supported in a given UA? Clients can try one of the new transferFunction enum values, and they should get a TypeError exception calling into decodingInfo(). So clients have a path to discovering that UAs don't "understand" a given transferFunction value.

The remaining question is how the client knows whether it must provide transferFunction, colorGamut, etc, or whether that information is derivable from the MIME type. Let me throw out a straw man.

We already have MediaCapabilitiesDecodingInfo.configuration which contains the configuration as understood by the UA, to allow clients to detect UA support for new fields in MediaConfiguration. One way to answer this question would be for UAs to "fill in" the configuration object with the information provided by the MIME type. E.g., if the client passed in {video: { contentType: 'video/webm; codecs="vp09.02.10.10.01.09.16.09.01"'; width: 1080; height: 720; bitrate: 10000; frameRate: 24 } }, the resulting configuration object would also include {...; hdrMetadataType: X; colorGamut: Y; transferFunction: Z } as derived from the MIME by the UA.

If we had this kind of API, clients which have a MIME-type that specifies HDR properties can discover that they don't need to specify the HDR properties separately. In supporting UAs, the HDR properties will be present in the resulting configuration. In non-supporting UAs, or with MIMEs that don't specify HDR properties, they know that they must provide the HDR properties separately (because those values are missing from the resulting configuration, and they have a way of determining whether the values they've derived are understood by the UA (via the TypeError above).

mounirlamouri · 2020-06-08T15:40:11Z

Removing the agenda label as I forgot to to that after the previous meeting.

I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d

dalecurtis · 2023-10-31T00:51:29Z

While this seems resolved via the WG and implemented by Safari, the spec text hasn't yet been updated. Chrome will implement reporting mismatched color spaces as unsupported like Safari does.

I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d

I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <[email protected]> Commit-Queue: Dale Curtis <[email protected]> Cr-Commit-Position: refs/heads/main@{#1217958}

I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <[email protected]> Commit-Queue: Dale Curtis <[email protected]> Cr-Commit-Position: refs/heads/main@{#1217958}

… by default., a=testonly Automatic update from web-platform-tests Prepare to enable HDR media capabilities by default. I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <[email protected]> Commit-Queue: Dale Curtis <[email protected]> Cr-Commit-Position: refs/heads/main@{#1217958} -- wpt-commits: f1f1925c4876c73f932d35e0a038b619b861ed83 wpt-pr: 42856

… by default., a=testonly Automatic update from web-platform-tests Prepare to enable HDR media capabilities by default. I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <kronchromium.org> Commit-Queue: Dale Curtis <dalecurtischromium.org> Cr-Commit-Position: refs/heads/main{#1217958} -- wpt-commits: f1f1925c4876c73f932d35e0a038b619b861ed83 wpt-pr: 42856 UltraBlame original commit: c8d7432b90a19fe34fe69215a157ffb4ddd8e459

… by default., a=testonly Automatic update from web-platform-tests Prepare to enable HDR media capabilities by default. I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <[email protected]> Commit-Queue: Dale Curtis <[email protected]> Cr-Commit-Position: refs/heads/main@{#1217958} -- wpt-commits: f1f1925c4876c73f932d35e0a038b619b861ed83 wpt-pr: 42856

… by default., a=testonly Automatic update from web-platform-tests Prepare to enable HDR media capabilities by default. I2S approved here: https://groups.google.com/a/chromium.org/g/blink-dev/c/0neM-5GDn8I/m/ymH32sDDAgAJ This cleans up an unnecessary feature flag and implements the rest of the feature per the current spec. Adds a test for mismatched color space information. w3c/media-capabilities#152 covers how we should reject mismatched color parameters, though a PR to add it to the spec hasn't yet been made. Safari has implemented this behavior though, so copy it. Bug: 1048045 Change-Id: I4fee62361c697922bb041724f4e4ac8c8e9d5c2d Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/4989038 Reviewed-by: Johannes Kron <kronchromium.org> Commit-Queue: Dale Curtis <dalecurtischromium.org> Cr-Commit-Position: refs/heads/main{#1217958} -- wpt-commits: f1f1925c4876c73f932d35e0a038b619b861ed83 wpt-pr: 42856 UltraBlame original commit: 04c64b3491fa73115ddc56ccc442cb7655e7bd2e

markafoltz · 2024-09-17T20:57:40Z

The PR updates the spec to reflect the implemented behavior in #152 (comment).

FWIW I think it's more logical to check this in the steps to validate a video configuration. However, an invalid configuration rejects the Promise returned by decodingInfo with a TypeError, and that's not what was implemented.

vi-dot-cpp added the agenda Topic should be discussed in a group call label Apr 16, 2020

mounirlamouri removed the agenda Topic should be discussed in a group call label Jun 8, 2020

chromium-wpt-export-bot mentioned this issue Oct 31, 2023

Prepare to enable HDR media capabilities by default. web-platform-tests/wpt#42856

Merged

chrisn added this to the V1 milestone Dec 15, 2023

chrisn added the pr-needed label Dec 15, 2023

chrisn added the TPAC2024 Topic for discussion at TPAC 2024 label Sep 13, 2024

markafoltz self-assigned this Sep 15, 2024

markafoltz added a commit that referenced this issue Sep 17, 2024

Address Issue #152.

38a53ca

markafoltz linked a pull request Sep 17, 2024 that will close this issue

Handle video profiles mismatched with colorGamut and transferFunction. #231

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Discuss how to reconcile colorGamut & transferFunction with ISO 23001-8:2016 #152

Discuss how to reconcile colorGamut & transferFunction with ISO 23001-8:2016 #152

vi-dot-cpp commented Mar 31, 2020

tidoust commented May 15, 2020

vi-dot-cpp commented May 18, 2020

chcunningham commented Jun 4, 2020

jernoble commented Jun 4, 2020

mounirlamouri commented Jun 8, 2020

dalecurtis commented Oct 31, 2023

markafoltz commented Sep 17, 2024 •

edited

Loading

Discuss how to reconcile colorGamut & transferFunction with ISO 23001-8:2016 #152

Discuss how to reconcile colorGamut & transferFunction with ISO 23001-8:2016 #152

Comments

vi-dot-cpp commented Mar 31, 2020

tidoust commented May 15, 2020

vi-dot-cpp commented May 18, 2020

chcunningham commented Jun 4, 2020

jernoble commented Jun 4, 2020

mounirlamouri commented Jun 8, 2020

dalecurtis commented Oct 31, 2023

markafoltz commented Sep 17, 2024 • edited Loading

markafoltz commented Sep 17, 2024 •

edited

Loading