Split streams into more features

[MattiasBuelens]

Prompted by the discussion in web-platform-tests/wpt#49829, I had a look at the current list of features in the streams group. Currently, there's only the "base" streams feature and a single async-iterable-streams extension for async iteration. However, there have been several additions over the years since we shipped BYOB support:

• Transferable streams, added to the standard in August 2020 and shipped in Chrome 87 and Firefox 103
• ReadableStream.from(asyncIterable), added to the standard in June 2023 and shipped in Firefox 117
• Transformer.cancel(), added to the standard in September 2023 but not yet shipping in any browser
• ReadableStreamBYOBReader.read(view, {min}), added to the standard in November 2023 and shipped in Firefox 134

I suggest we add the following new features, and move them out of the base streams feature:

• transferable-streams
• readablestream-from
• transformstream-transformer-cancel
  • I've added a new BCD feature for this: Add transformer.cancel method to TransformStream constructor mdn/browser-compat-data#25519
• readablestreambyobreader-read-min

[MattiasBuelens]

@jcscottiii FYI

[captainbrosset]

Given the differences in browser support and ship dates, I agree that it makes sense to split Streams into multiple sub-features. These seem separate enough to me.

That said, I don't have enough experience with streams to know whether that's also how developers think of them.
The key aspect we use to define features on this repo is developer perception.
Some of the questions we try to answer to split things up might be:

• Do web developers think of these features independently from each other?
• Can they use streams already without these later additions?
• Are these later additions improvements over the base feature?
• Is there evidence of these additions being talked about on the web (StackOverflow, blogs, etc.)?

Let us know your thoughts. I'm not saying this isn't the right way to split, merely just leaving thought to provoke conversation.

Beyond that, I made a few tiny description proposals. Also, this should probably be merged again on the latest main branch and dist files should be re-generated.

[MattiasBuelens]

Can they use streams already without these later additions?

Yes, streams have been usable for quite a while, even before these recent additions.

• 2016 - the year of web streams (Jake Archibald) shows that back in 2016, streams were already very usable.
• Streams - The definitive guide on web.dev talks about all the APIs available in the base feature, how to get a stream from fetch(), which other Web APIs provide streams (such as Encoding Streams and Compression Streams)

Are these later additions improvements over the base feature?

Yes, these are all additive features. Generally, the objective has been to make streams easier to use, and better integrated with various parts of the Web Platform:

• Transferable streams enables streams to be passed around between contexts (e.g. main thread and workers), similar to how ArrayBuffers are transferable. You could have done this yourself before (by manually postMessage()ing each chunk), but this makes it much easier to correctly handle backpressure, errors and cancellation.
• ReadableStream.from() makes it easier to create a stream from other data sources, e.g. from an array. You could have done this yourself before (by manually constructing a stream and calling controller.enqueue()), but this makes it a one-liner in most cases.
• Transformer.cancel() adds a way to respond to cancellation for a user-defined TransformStream. You could have done this before with a custom ReadableStream and WritableStream, but the new transformer method is much better at handling all possible edge cases.
• ReadableStreamBYOBReader.read(view, {min}) extends the existing read(view) method. You could have done this before by manually calling read(view) in a loop until the view is sufficiently filled (like in this example), but the new min option makes this easier and enables potential optimizations in the future.

Do web developers think of these features independently from each other?
Is there evidence of these additions being talked about on the web (StackOverflow, blogs, etc.)?

• Transferable streams
  • This is an enabler for WebRTC Insertable Streams, allowing developers to run their processing in a worker thread.
    • In-browser media compositing via WebRTC Insertable Streams API (Yakup Çilesiz)
    • WebRTC samples Breakout Box crop
• ReadableStream.from()
  • Node.js added Readable.toWeb() to convert a Node.js Readable to a Web ReadableStream. With ReadableStream.from(), there's now a more standardized way to do this conversion.
  • Developers on StackOverflow are interested in polyfilling this method.
  • It makes for much more concise example code for streams, see Deno docs for @std/streams.
• Transformer.cancel
  • This arose from a Deno developer working on their TextDecoderStream implementation needing a reliable way to clean up their native decoder when the stream is cancelled or aborted.
  • Developers on StackOverflow are working around the lack of Transform.cancel using an extra wrapper WritableStream. However, as explained in that answer, this workaround doesn't handle all edge cases properly.
• ReadableStreamBYOBReader.read(view, {min})
  • This arose from a Cloudflare developer wanting to provide a more optimized method to read from a byte stream, when the user knows that they'll need at least N amount of bytes from the stream before they can continue useful work.

[captainbrosset]

This looks good to me. Thank you for providing the detailed rationale about these sub-features.

I don't think this needs to block this PR, but I do have one follow-up question: should the parent streams feature be Baseline Widely Available then?
Looking at that feature, it uses compute_from with the api.ReadableStreamBYOBReader BCD key, which makes it not available on Safari. But, reading the Streams guide you provided, it feels like the streams feature should be baseline if the base use cases have been available for a while already.

[MattiasBuelens]

I don't think this needs to block this PR, but I do have one follow-up question: should the parent streams feature be Baseline Widely Available then?
Looking at that feature, it uses compute_from with the api.ReadableStreamBYOBReader BCD key, which makes it not available on Safari. But, reading the Streams guide you provided, it feels like the streams feature should be baseline if the base use cases have been available for a while already.

Personally, I would like to see BYOB become a separate feature. It's a bit weird how e.g. Compression Streams (which builds upon Streams) is Baseline Newly Available, while Streams is only Baseline Limited Available.

There was a discussion regarding BYOB as part of #2358. The decision there was to keep BYOB as part of the base feature, in order to match the status quo on https://caniuse.com/streams. We could consider splitting up the feature in both Baseline and caniuse, but I don't know what the impact of such a split would be on developers using this feature data today. 🤷‍♂️