Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option to download folders as tar #308

Closed
abe1242 opened this issue Dec 2, 2023 · 6 comments
Closed

Option to download folders as tar #308

abe1242 opened this issue Dec 2, 2023 · 6 comments

Comments

@abe1242
Copy link

abe1242 commented Dec 2, 2023

Specific Demand

zip is really slow when sending large folders with multiple files, since it has to do compression.

Having an option to download as tar (without any compression like gz, bz xz, etc.) would be the better option.

@abe1242 abe1242 changed the title tar option would be nice Option to download folders as tar Dec 2, 2023
@sigoden
Copy link
Owner

sigoden commented Dec 2, 2023

If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot.

Dufs is not prepared to support two compression formats, just like github has always only supported zip.

Dufs supports downloading compressed streams. The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth.

@sigoden sigoden closed this as completed Dec 2, 2023
@abe1242
Copy link
Author

abe1242 commented Dec 2, 2023

The compression algorithm won't have a huge impact on download times. The biggest impact has always been bandwidth.

In my case bandwidth is not the problem. I can send files using dufs with an average speed of 40 MB/s. Whereas, for a folder it around 6 MB/s. Tried to download the same folder with the .tar option in miniserve and I got close to 40 MB/s.

If only one compression format is supported, zip is the only choice. Because zip is the most versatile, the evidence is that windows can directly open zip, but other formats cannot.

I do understand that the universal support of zip makes it a better default. You could implement tar by providing a flag like --archive-format which can be set to tar if needed, and by default it is set to zip.

Correction: speed of zip is around 3.5 MB/s when I tested it right now.

@sigoden
Copy link
Owner

sigoden commented Dec 2, 2023

image
The download speed of zip stream can reach 20M.
If there are many small files, the speed may be slower.

This speed is acceptable.

@abe1242
Copy link
Author

abe1242 commented Dec 2, 2023

My phone has the computing power of a potato so that explains the slow speed of zip. I guess I'm gonna have to stick to miniserve for sending folders and dufs for anything else.

Thanks for the response anyway

@sigoden
Copy link
Owner

sigoden commented Dec 14, 2023

#319 supports --compress: none which is similar to tar and is extremely fast. you may try it.

@abe1242
Copy link
Author

abe1242 commented Dec 17, 2023

Tried it on my PC and it works really well. Getting 170M+ speeds whereas with compression it is 20M. Haven't tried it on my phone, but I'm hoping it will be much faster than before.
Thank you so much for working on this. This was much needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants