Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add progress information #115

Open
K0-RR opened this issue Nov 1, 2022 · 3 comments
Open

Add progress information #115

K0-RR opened this issue Nov 1, 2022 · 3 comments

Comments

@K0-RR
Copy link

K0-RR commented Nov 1, 2022

I'm currently compressing about 2500 images. There is no ETA, % of completion. I have no idea if it's gonna take a few hours or many days. Please add at least something like this
Processed 135 (100 MB) out of 500 files (900 MB)

@fhanau
Copy link
Owner

fhanau commented Nov 3, 2022

This is similar to your other issue, but I think it may be out of scope. Being able to display progress would require a careful design so it works well for a number of different use cases. I consider the folder support of ECT to be a convenience feature and not its main function, so adding progress information to see how long it takes to compress one big folder is not what I plan to focus on.
If you are using ECT on larger amounts of data, I would recommend to use a script to process files one at a time, which allows you to print progress or any additional information as you go through the folder and makes it more flexible. When compressing larger amounts of data with a high compresssion setting, the run time can certainly be high, so you can run the program on a subset of the files or with a lower setting (e.g. -3) first to get a sense of how long it might take.

@Anasxrt
Copy link

Anasxrt commented Jul 4, 2023

I use xargs to help,

find . -type f -name "*.png" -o -name "*.PNG" -o -name "*.jpg" -o -name "*.JPG" | xargs -n1 ect -3 -recurse -strip -keep --mt-file

but I have not figure out yet, to show the file names in line before optimize result output

@taskhawk
Copy link

taskhawk commented Feb 2, 2024

I use this:

find . -type f -iname "*.png" -o -iname "*.jpg" -print0 | sort -z | xargs -0r -n 1 -P $(nproc) -I '{}' sh -c 'OUT=$(ect -3 -strip -keep "$1" | grep -v "^Processed") && echo "$1  -->  $OUT" || echo "Error processing $1"' sh '{}'

Files are delimited by a null character to avoid any funny business with weird characters in filenames. nproc returns the number of logical processors in your system to maximize performance, adjust as necessary. From my short testing it appears it's more efficient to run more single-threaded processes than fewer multi-threaded processes. I sort the results of find to help give a sense of progress but mainly so it can help me locate approximately which file caused issues (with an error margin +/- of the value used for -P) in case the error isn't catched by || echo "Error processing $1". The output for each file is converted to a single line because the multiple processes running at the same time could mix their multi-line output making it difficult to know to which result corresponds to which file.

However, I find the command a monstrosity and I think it would be better if ect just showed what file(s) it was working on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants