Skip download for existing resources

[razvanstatescu]

Hi, just found your module and it looks awesome. Good job for making it! I just have a quick question: I'm trying to skip download for existing files (don't make the request). It is possible right now to skip doing a request for specific files using a custom plugin?

I tried doing something like

registerAction("beforeRequest", async ({ resource, requestOptions }) => {
      const filename = path.join(absoluteDirectoryPath, resource.getFilename());
      if (fs.existsSync(filename)) {
        console.log("skip ", resource.getFilename());
        throw new Error("file exists");
      }
    });

And add ignoreErrors: true but it looks like the script stop working after the first error is thrown.

Thanks a lot for the help!

[s0ph1e]

Hi @razvanstatescu 👋

Thank you for your feedback!

If you're downloading website to new directory (and not using existing directory plugin or something similar) it should skip already downloaded filed by default. This module checks if url was already downloaded and reuse already downloaded resource reference if same url appears in other places. E.g. if you have 10 links to ./some-image.png - it will be downloaded only once but link will be changed in 10 places.

I hope it answers your question.