Cleanup CLI command logic deletes all files using "Default File Download Template"

[imorland]

Related: #360 #352 #333

Reported on discuss: https://discuss.flarum.org/d/4154-friendsofflarum-upload-the-intelligent-file-attachment-extension/2234

I think the cleanup logic is faulty. I just executed it for the first time with --force and I didn't care to check if the files are indeed used (I don't know how to do that anyway, it just lists a URL but I can't possibly know/remember if such a file might be used anywhere on the forum or not...). I opened one of the important old discussion of the forum with many uploads inside and those uploads that were local and use the Download template are not working anymore and I don't keep the files anymore 😕 This is really bad... I can try restoring them from a backup of the forum though but I'm wondering on what other places it may have deleted files, e.g. Amazon where I don't have backups... Really having bad cold chills now, hopefully I haven't screwed my forum uploads for the last three years.

P.S. Warning!

I can confirm the cleanup logic is faulty. When a file was uploaded using the "Default File Download Template" in FoF Upload, the cleanup logic cannot map those files and would assume they are not used, while they are in fact used in posts, and would subsequently delete them from the corresponding storage, in my case local and AWS S3. "Fortunately" I deleted only files between January and December 2022 on my forum, but that's still a lot of files and I broke some of the most important discussions with audio files on my forum and it's a music related forum where people upload their own recordings, etc. To say it's disaster would be an understatement, since I don't have backups and we lost important content forever. I'll try to not be dramatic and just accept that sh1t happens from time to time and it was my turn today, not the end of world after all 😕 I probably lost some trust in my users since I have never screwed up that much and they trusted me that the forum is reliable for uploading files. But I would strongly advise people to be EXTREMELY careful before using the cleanup logic. Always make a backup not only of your forum, but also of all the files on external services such as AWS S3 because the cleanup can delete them too.

P.P.S. Is there any log of the cleanup command that I can find somewhere to at least list all the deleted files, so that I can warn people about the deletions?

[NathanSweet]

Spammers are composing a new post, uploading files to my server, not posting, and then using URLs to those files in their spam emails. I can't be giving spammers free hosting! Now that the spammers know about this exploit and are abusing it, all fof-upload users are at risk.

Is it safe to uncomment MapFilesCommand and run it? I'm using only adapters: default file download, just URL, and complete image preview template.

[NathanSweet]

I removed my site from the showcase since I expect now that spammers know Flarum can be abused, having a collection of sites to exploit is convenient.

[github-actions]

This issue has been automatically closed because it received no activity for three months. If you think it was closed by accident, please leave a comment. If you are running into a similar issue on the latest version, please open a new issue. Thank you.

[tipa]

@imorland @DavideIadeluca why is this issue closed? Is this change permanent?
The command is still documented here

[DavideIadeluca]

@tipa Thanks for highlighting. This issue got closed accidentally.