Events order when parsing file

[Hyodori04]

When read file, busboy occur file.on('data') events before ahead file file.on('end) events. I don't think this is handled by highWaterMark or fileHwm.

BusBoy.on("file", (fieldName: string, file: Readable, info: FileInfo) => {
      file.on("data", (chunk: Buffer) => {
            // implementation
      });

      file.on("end", () => {
        // implementation
      });

      file.on("limit", () => {
          // implementation
      });
    });

I think that this is not happen always. I think this is related to each file size.

Is there any method that I can confirm order of file reading between end -> data?

[mscdex]

I'm not sure what you're saying. file is just a normal node Readable stream (aside from the custom limit-related event and property), so all of the normal node stream behaviors apply. If you're seeing 'data' events after 'end'/'close' events, then that's a node.js bug.

[Hyodori04]

I mean when reading multiple files. Before first files occur 'end' events, event "data" of secods file occurs. This order is not handled by busboy?

[mscdex]

I would say that would be due to node delaying 'end'/'close' events until at least the next tick of the event loop and/or #346.

[Hyodori04]

Thanks for fast anwser. I will find some way to solve this issue.

[Hyodori04]

I think this timing issue not happens randomly. It's related files size. (If it's related only even loop, I think it should happens randomly) Only certain size of file group generate this problem. Is there any way to handle this issue in busboy?

[Hyodori04]

I have solved this issue by using pause and resume function in readalbe objects

    let index = 0;
    const fileList: Readable[] = [];

    BusBoy.on("file", (fieldName: string, file: Readable, info: FileInfo) => {
      let isFirst = true;
      const ownerIndex = index;
      index += 1;

      file.pause();
      fileList.push(file);

      const isFirstFile = ownerIndex === 0;

      if (isFirstFile) {
        const targetFile = fileList.shift();
        targetFile!.resume();
      }

      file.on("data", (chunk: Buffer) => {
         // implementation
      });

      file.on("end", () => {
         // implementation

        const nextFile = fileList.shift();
        if (nextFile) {
          nextFile.resume();
          return;
        }
      });
    });