TypeError when processing data through DecompressionStream: Unable to decompress chunk #3294

aplefull · 2025-01-18T13:48:08Z

Summary

Reduction comes from 0x40.mon.im (photosensitive epilepsy warning!).

This website loads assets in a zip archive and uses decompression streams to unpack it. In Ladybird it results in a
[TypeError] Unable to decompress chunk: Reached end-of-stream without collecting the required number of bits and prevents from loading songs there.

Operating system

Linux

Steps to reproduce

Run reduced example.

Expected behavior

No errors, readable should be locked by the end of code execution.

Actual behavior

Errors, readable is unlocked.

URL for a reduced test case

N/A

HTML/SVG/etc. source for a reduced test case

<!DOCTYPE html>
<body>
  <script defer>
    const main = async () => {
      const zipData =
        "data:@file/zip;base64,UEsDBBQAAAAIAF0px0TfyjcygwAAALQAAAAIAAAAaW5mby54bWxNjj0OwjAMhWeQuIMvQN2BCVmR2FjYeoFQ0iaiiaPGlXp8ErN0s/y+90MhTWwu5xMlG515bq7AK+zwIGwPVewmnlczeAf9futBocHZSPiXlPq4Mq4hS+B0iIErWIj14gkKp7kQHkA1LiF9jRfJd8SW76u5ey88l8zSjVxrGlJZQl37A1BLAQI/ABQAAAAIAF0px0TfyjcygwAAALQAAAAIACQAAAAAAAAAIAAAAAAAAABpbmZvLnhtbAoAIAAAAAAAAQAYAEBZsAu7gc8Bfo4xT8Ou2AG2ON55fKvPAVBLBQYAAAAAAQABAFoAAACpAAAAAAA=";
      const dataBlob = await fetch(zipData).then((response) => response.blob());

      const workerScript = document.querySelector(
        "script[type='worker']",
      ).textContent;

      const script = URL.createObjectURL(
        new Blob([workerScript], { type: "text/javascript" }),
      );

      const readable = new ReadableStream({
        start() {
          this.chunkOffset = 0;
        },
        async pull(controller) {
          const { size } = readable;
          const { chunkOffset } = this;
          const chunkSize = 524288;

          let arrayBuffer = await dataBlob.arrayBuffer();
          controller.enqueue(new Uint8Array(arrayBuffer));

          if (chunkOffset + chunkSize > size) {
            controller.close();
          } else {
            this.chunkOffset += chunkSize;
          }
        },
      });

      async function onMessage({ data }, workerData) {
        const { type, messageId } = data;
        const { worker } = workerData;

        if (type == "pull") {
          const message = {
            type: "data",
            value: undefined,
            done: true,
            messageId,
          };
          worker.postMessage(message);
        }
      }

      const worker = new Worker(new URL(script));
      const workerData = { worker };
      worker.addEventListener("message", (event) =>
        onMessage(event, workerData),
      );

      const message = {
        type: "start",
        readable: readable,
      };

      try {
        worker.postMessage(message, [readable]);
      } catch {
        message.readable = null;
        worker.postMessage(message);
      }

      // Ends up being true in Chrome and Firefox, false in Ladybird
      console.log(readable.locked);
    };

    main();
  </script>
  <script type="worker">
    let pendingResolve = null;

    addEventListener("message", ({ data: e }) => {
        const { type, value, done } = e;

        if (type == "start") {
            const readableStream = new ReadableStream({
            async pull(controller) {
                const promise = new Promise(resolve => pendingResolve = resolve);
                postMessage({ type: "pull", messageId: 0 });

                const { value, done } = await promise;
                controller.enqueue(value);
                if (done) controller.close();
            }
            });

            return readableStream
            .pipeThrough(new TransformStream({
                transform(chunk, controller) {
                controller.enqueue(chunk);
                }
            }))
            .pipeThrough(new DecompressionStream("deflate-raw"))
            .pipeThrough(new TransformStream());
        } else if (type == "data") {
            pendingResolve({ value: new Uint8Array(value), done });
            pendingResolve = null;
        }
    });
  </script>
</body>

Log output and (if possible) backtrace

12561.181 WebContent(36602): ResponsePrototype::blob()
12561.181 WebContent(36602): WindowGlobalMixin::document_getter()
12561.181 WebContent(36602): DocumentPrototype::query_selector("script[type='worker']")
12561.181 WebContent(36602): NodePrototype::text_content_getter()
12561.181 WebContent(36602): BlobConstructor::construct([object Array], [object Object])
12561.182 WebContent(36602): DOMURLConstructor::create_object_url([object Blob])
12561.182 WebContent(36602): ReadableStreamConstructor::construct([object Object])
12561.182 WebContent(36602): DOMURLConstructor::construct("blob:file:///ee5bc67a-3cbd-498f-99b1-467d42336603")
12561.182 WebContent(36602): WorkerConstructor::construct([object DOMURL])
12561.182 WebContent(36602): DOMURLPrototype::to_string()
12561.183 Ladybird(36587): Destroying Thread ""(131831536027328) while it is still running undetached!
12561.183 WebContent(36602): EventTargetPrototype::add_event_listener("message", [object ECMAScriptFunctionObject])
12561.183 WebContent(36602): WorkerPrototype::post_message([object Object], [object Array])
12561.183 WebContent(36602): WorkerPrototype::post_message0([object Object], [object Array])
12561.183 WebContent(36602): DOMExceptionPrototype::message_getter()
12561.183 WebContent(36602): THROW! Cannot transfer type
12561.183 WebContent(36602): -> 
12561.183 WebContent(36602): -> main @ file:///home/aplefull/Downloads/streams.html:62,27
12561.183 WebContent(36602): -> main
12561.183 WebContent(36602): -> 
12561.183 WebContent(36602): -> 
12561.183 WebContent(36602): -> 
12561.183 WebContent(36602): WorkerPrototype::post_message([object Object])
12561.183 WebContent(36602): WorkerPrototype::post_message1([object Object])
12561.183 WebContent(36602): ReadableStreamPrototype::locked_getter()
12561.183 WebContent(36602): (js log) false
12561.184 WebContent(36602): BlobPrototype::array_buffer()
12561.184 WebContent(36602): ReadableStreamDefaultControllerPrototype::enqueue([object Uint8Array])
12561.234 WebContent(36602): MessageEventPrototype::data_getter()
12561.234 WebContent(36602): WorkerPrototype::post_message([object Object])
12561.234 WebContent(36602): WorkerPrototype::post_message1([object Object])
12561.235 WebWorker(36618): Unhandled JavaScript exception (in promise): [TypeError] Cannot close an errored stream
12561.235 WebWorker(36618):     at <unknown>
    at <unknown>

12561.235 WebWorker(36618): Unhandled JavaScript exception (in promise): [TypeError] Unable to decompress chunk: Reached end-of-stream without collecting the required number of bits
12561.235 WebWorker(36618):     at <unknown>
    at transform (blob:file:///ee5bc67a-3cbd-498f-99b1-467d42336603:22:35)
    at <unknown>
    at <unknown>
    at pull
    at pull
    at <unknown>
    at <unknown>

Screenshots or screen recordings

No response

Build flags or config settings

No response

Contribute a patch?

I’ll contribute a patch for this myself.

The text was updated successfully, but these errors were encountered:

trflynn89 · 2025-01-18T22:54:28Z

I think the underlying issue is this bit in your repro:

try {
  worker.postMessage(message, [readable]);
} catch {
  message.readable = null;
  worker.postMessage(message);
}

We aren't yet able to transfer ReadableStream, so postMessage(message, [readable]) throws an exception. If I force Firefox to use the catch branch, it also logs false.

So (at least as a start), it seems this issue will be to resolve this FIXME:

ladybird/Libraries/LibWeb/HTML/StructuredSerialize.cpp

Line 1185 in 3bfb053

// FIXME: Handle transferring objects with [[Detached]] internal slot.

shannonbooth · 2025-01-18T23:04:04Z

See #3129 which makes a start at working towards ReadableStream transferable and links the main set of changes which are still getting ironed out. It's all implemented, but I believe we have some issues in MessagePort which blocks that.

shannonbooth · 2025-01-20T21:45:09Z

I've been trying to dig into this a little more, I think the main blocker for merging: shannonbooth@77c937a and shannonbooth@5c5f250 to implement ReadableStream transfer is in MessagePort.

Specifically, I think this type of requirement appears to potentially be a/the source of the problem:

ladybird/Libraries/LibWeb/HTML/MessagePort.cpp

Line 126 in 026fc6c

    
           // FIXME 2. Move all the tasks that are to fire message events in dataHolder.[[PortMessageQueue]] to the port message queue of value,

I find it quite difficult to understand the requirements on the spec when it comes to MessagePort, but I'm beginning to think the only way that we could hope to implement these types of requirements is by a handshake between the two entangled message ports to flush queued messages before completely tearing down the connection on a close().

AtkinsSJ added bug Something isn't working reduction of web content Issue has a simplified reduction based on real-world web content. has repro We have a way to reproduce this bug. labels Jan 20, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TypeError when processing data through DecompressionStream: Unable to decompress chunk #3294

TypeError when processing data through DecompressionStream: Unable to decompress chunk #3294

aplefull commented Jan 18, 2025

trflynn89 commented Jan 18, 2025

shannonbooth commented Jan 18, 2025 •

edited

Loading

shannonbooth commented Jan 20, 2025 •

edited

Loading

TypeError when processing data through DecompressionStream: Unable to decompress chunk #3294

TypeError when processing data through DecompressionStream: Unable to decompress chunk #3294

Comments

aplefull commented Jan 18, 2025

Summary

Operating system

Steps to reproduce

Expected behavior

Actual behavior

URL for a reduced test case

HTML/SVG/etc. source for a reduced test case

Log output and (if possible) backtrace

Screenshots or screen recordings

Build flags or config settings

Contribute a patch?

trflynn89 commented Jan 18, 2025

shannonbooth commented Jan 18, 2025 • edited Loading

shannonbooth commented Jan 20, 2025 • edited Loading

shannonbooth commented Jan 18, 2025 •

edited

Loading

shannonbooth commented Jan 20, 2025 •

edited

Loading