diff --git a/index.bs b/index.bs index 3d15bda..42b7074 100644 --- a/index.bs +++ b/index.bs @@ -142,10 +142,10 @@ The writeEncodedData algorithm is given a |rtcObject| as 1. Set |rtcObject|.`[[lastReceivedFrameCounter]]` to |frame|`[[counter]]`. 1. Let |data| be |frame|.`[[data]]`. 1. Let |serializedFrame| be [$StructuredSerializeWithTransfer$](|frame|, « |data| »). -1. Let |frameCopy| be [$StructuredDeserialize$](|serializedFrame|, |frame|'s [=relevant realm=]). -1. Enqueue |frameCopy| for processing as if it came directly from the encoded data source, by running one of the following steps: - * If |rtcObject| is a {{RTCRtpSender}}, enqueue |frameCopy| to |rtcObject|'s packetizer, to be processed [=in parallel=]. - * If |rtcObject| is a {{RTCRtpReceiver}}, enqueue |frameCopy| it to |rtcObject|'s decoder, to be processed [=in parallel=]. +1. Let |transferredFrame| be [$StructuredDeserializeWithTransfer$](|serializedFrame|, |frame|'s [=relevant realm=]). +1. Enqueue |transferredFrame| for processing as if it came directly from the encoded data source, by running one of the following steps: + * If |rtcObject| is a {{RTCRtpSender}}, enqueue |transferredFrame| to |rtcObject|'s packetizer, to be processed [=in parallel=]. + * If |rtcObject| is a {{RTCRtpReceiver}}, enqueue |transferredFrame| it to |rtcObject|'s decoder, to be processed [=in parallel=]. 1. Return [=a promise resolved with=] undefined. On sender side, as part of [$readEncodedData$], frames produced by |rtcObject|'s encoder MUST be enqueued in |rtcObject|.`[[readable]]` in the encoder's output order. @@ -371,7 +371,7 @@ dictionary RTCEncodedVideoFrameMetadata {

An identifier for the encoded frame, monotonically increasing in decode order. Its lower - 16 bits match the frame_number of the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[?AV1-RTP-SPEC]], if present. + 16 bits match the frame_number of the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[AV1-RTP-SPEC]], if present. Only present for received frames if the Dependency Descriptor Header Extension is present.

@@ -381,7 +381,7 @@ dictionary RTCEncodedVideoFrameMetadata {

List of frameIds of frames this frame references. - Only present for received frames if the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[?AV1-RTP-SPEC]] is present. + Only present for received frames if the AV1 Dependency Descriptor Header Extension defined in Appendix A of [[AV1-RTP-SPEC]] is present.

@@ -451,7 +451,7 @@ dictionary RTCEncodedVideoFrameOptions { // New interfaces to define encoded video and audio frames. Will eventually // re-use or extend the equivalent defined in WebCodecs. -[Exposed=(Window,DedicatedWorker), Serializable] +[Exposed=(Window,DedicatedWorker), Serializable, Transferable] interface RTCEncodedVideoFrame { constructor(RTCEncodedVideoFrame originalFrame, optional RTCEncodedVideoFrameOptions options = {}); readonly attribute RTCEncodedVideoFrameType type; @@ -460,7 +460,7 @@ interface RTCEncodedVideoFrame { }; -### Constructor ### {#RTCEncodedVideoFrame-members} +### Constructor ### {#RTCEncodedVideoFrame-constructor}
constructor() @@ -506,11 +506,13 @@ interface RTCEncodedVideoFrame { {{RTCEncodedVideoFrameMetadata/mimeType}}. For SVC, each spatial layer is transformed separately. -

+

+

Since packetizers may drop certain elements, e.g. AV1 temporal delimiter OBUs, the input to an receive-side transform may be different from the output of a send-side transform. -

+

+

The following table gives a number of examples:

@@ -580,26 +582,41 @@ interface RTCEncodedVideoFrame { -### Serialization ### {#RTCEncodedVideoFrame-serialization} +### Transfer and Serialization ### {#RTCEncodedVideoFrame-serialization} + +{{RTCEncodedVideoFrame}} objects are [=serializable objects=] and [=transferable objects=]. -{{RTCEncodedVideoFrame}} objects are serializable objects [[HTML]]. Their [=serialization steps=], given |value|, |serialized|, and |forStorage|, are: 1. If |forStorage| is true, then throw a {{DataCloneError}}. -1. Set |serialized|.`[[type]]` to the value of |value|.{{RTCEncodedVideoFrame/type}} +1. Set |serialized|.`[[type]]` to the value of |value|.{{RTCEncodedVideoFrame/type}}. 1. Set |serialized|.`[[metadata]]` to an internal representation of |value|'s metadata. -1. Set |serialized|.`[[data]]` to |value|.`[[data]]` +1. Set |serialized|.`[[data]]` to a copy of |value|.`[[data]]`. Their [=deserialization steps=], given |serialized|, |value| and |realm|, are: -1. Set |value|.{{RTCEncodedVideoFrame/type}} to |serialized|.`[[type]]` -1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]` +1. Set |value|.{{RTCEncodedVideoFrame/type}} to |serialized|.`[[type]]`. +1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]`. 1. Set |value|.`[[data]]` to |serialized|.`[[data]]`. +Their [=transfer steps=], given |value| and |dataHolder|, are: + +1. Set |dataHolder|.`[[type]]` to the value of |value|.{{RTCEncodedVideoFrame/type}}. +1. Set |dataHolder|.`[[metadata]]` to an internal representation of |value|'s metadata. +1. Set |dataHolder|.`[[data]]` to |value|.`[[data]]`. +1. Unset |value|.`[[data]]`. + +Their [=transfer-receiving steps=], given |dataHolder| and |value|, are: + +1. Set |value|.{{RTCEncodedVideoFrame/type}} to |dataHolder|.`[[type]]` +1. Set |value|'s metadata to the platform object representation of |dataHolder|.`[[metadata]]`. +1. Set |value|.`[[data]]` to |dataHolder|.`[[data]]`. +

The internal form of a serialized RTCEncodedVideoFrame is not observable; -it is defined chiefly so that it can be used with frame cloning in the -[$writeEncodedData$] algorithm and in the {{WindowOrWorkerGlobalScope/structuredClone()}} operation. +transfer is used to hand off frames in the +[$writeEncodedData$] algorithm, and serialization is used in the +{{WindowOrWorkerGlobalScope/structuredClone()}} operation. An implementation is therefore free to choose whatever method works best.

@@ -681,7 +698,7 @@ dictionary RTCEncodedAudioFrameOptions { RTCEncodedAudioFrameMetadata metadata; }; -[Exposed=(Window,DedicatedWorker), Serializable] +[Exposed=(Window,DedicatedWorker), Serializable, Transferable] interface RTCEncodedAudioFrame { constructor(RTCEncodedAudioFrame originalFrame, optional RTCEncodedAudioFrameOptions options = {}); attribute ArrayBuffer data; @@ -689,7 +706,7 @@ interface RTCEncodedAudioFrame { }; -### Constructor ### {#RTCEncodedAudioFrame-members} +### Constructor ### {#RTCEncodedAudioFrame-constructor}
constructor() @@ -806,20 +823,32 @@ interface RTCEncodedAudioFrame {
-### Serialization ### {#RTCEncodedAudioFrame-serialization} +### Transfer and Serialization ### {#RTCEncodedAudioFrame-serialization} + +{{RTCEncodedAudioFrame}} objects are [=serializable objects=] and [=transferable objects=]. -{{RTCEncodedAudioFrame}} objects are serializable objects [[HTML]]. Their [=serialization steps=], given |value|, |serialized|, and |forStorage|, are: 1. If |forStorage| is true, then throw a {{DataCloneError}}. 1. Set |serialized|.`[[metadata]]` to an internal representation of |value|'s metadata. -1. Set |serialized|.`[[data]]` to |value|.`[[data]]` +1. Set |serialized|.`[[data]]` to a copy of |value|.`[[data]]` Their [=deserialization steps=], given |serialized|, |value| and |realm|, are: 1. Set |value|'s metadata to the platform object representation of |serialized|.`[[metadata]]` 1. Set |value|.`[[data]]` to |serialized|.`[[data]]`. +Their [=transfer steps=], given |value| and |dataHolder|, are: + +1. Set |dataHolder|.`[[metadata]]` to an internal representation of |value|'s metadata. +1. Set |dataHolder|.`[[data]]` to |value|.`[[data]]`. +1. Unset |value|.`[[data]]`. + +Their [=transfer-receiving steps=], given |dataHolder| and |value|, are: + +1. Set |value|'s metadata to the platform object representation of |dataHolder|.`[[metadata]]`. +1. Set |value|.`[[data]]` to |dataHolder|.`[[data]]`. + ## Interfaces ## {#RTCRtpScriptTransformer-interfaces}
 [Exposed=DedicatedWorker]
@@ -867,9 +896,9 @@ The