Beginning of file getting truncated under production conditions.

[charlesdeuter]

Getting a corrupted file while running code on the production server.

Opened it in hex editor to compare with the version that I received locally which was fine. Looks like a couple kb is truncated from the beginning of the file.

Here is the code.

    let latestFileName;

    let ftp = new PromiseFtp();

    return fs.mkdirpAsync(tempDir)
      .then(() => ftp.connect({/*host,user, pw, port*/}))
      .then(() => ftp.list('/'))
      .then((list) => {
        latestFileName = _(list)
          .sortBy('name')
          .last()
          .name;

        return ftp.get(latestFileName);
      })
      .then((readStream__) => {
        return new Promise((resolve, reject) => {
          const tempFileLoc = path.join(tempDir, latestFileName);

          let __writeStream = fs.createWriteStream(tempFileLoc);
          readStream__.once('close', resolve);
          readStream__.once('error', reject);
          readStream__.pipe(__writeStream);
        });
      })
      .finally(() => ftp.end())

The file itself is around 165mbs

[zacronos]

Hmm, that's unexpected -- I have used this module in production without seeing that happen. Since get is a simple promisified passthrough to the underlying ftp module, there's not a lot of room to screw this up -- and looking at the code, I don't see anything that could be causing this. (That doesn't mean the problem isn't in promise-ftp, just that I don't see it.)

Just to check, what version of promise-ftp are you using?

If you're using the latest version (v1.3.3), since I can't reproduce this issue and I can't find the problem by code inspection, as I see it there are 2 options for troubleshooting:

1. You can email me to provide me credentials to an FTP server and file where you see this happening (ideally a dev/test server you have set up for this purpose, rather than anything production).
2. You can do some troubleshooting at my suggestion, and we can go back and forth to solve the issue. If you want to do this, the first step would be to try using the same flow with the bare ftp module rather than promise-ftp and see whether or not you get the same results, and let me know.

Let me know how you want to proceed.

[charlesdeuter]

I'll give this a shot with the vanilla ftp module. I will get back to you with the results when I can, thanks!

[charlesdeuter]

version is 3.10.8, btw.

[zacronos]

Could you double-check the promise-ftp version? The latest is only v1.3.3

[charlesdeuter]

oops, sorry version 1.3.2.

[warncke]

I encountered a similar issue with getting empty buffers on small files.

I tracked this down to ftp/lib/connection.js:617 - the socket starts out paused but as soon as the callback is called it is resumed.

In normal callback mode the callback function would setup the stream reading events and everything would be ready to go when sock.resume() is called.

With promise-ftp the then function after get ends up getting called after the sock.resume() and after a data event has already occurred.

[charlesdeuter]

Sorry for the long delay. I can confirm the vanilla ftp module worked as expected.