Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

duplicacy command startup is extremely slow, does it do some network connection check? #658

Open
realzhang opened this issue Oct 12, 2023 · 8 comments

Comments

@realzhang
Copy link

realzhang commented Oct 12, 2023

Just type the duplicacy command and press enter (to print out the help info), the comman will take about 3 min to show the help info.
I've tried the precompiled bin and the binary file compiled from downloaed source. Both are very slow.
I do not understand why, but it seems that duplicacy does some network connection which last about 2 min.
OS is almalinux with kernel 5.4.17-2136.307.3.1.el8uek.x86_64 GNU/Linux

~ time ./duplicacy_linux_x64_3.2.3 -h
NAME:
duplicacy - A new generation cloud backup tool based on lock-free deduplication

USAGE:
duplicacy [global options] command [command options] [arguments...]

VERSION:
3.2.3 (254953)

COMMANDS:
init Initialize the storage if necessary and the current directory as the repository
backup Save a snapshot of the repository to the storage
restore Restore the repository to a previously saved snapshot
list List snapshots
check Check the integrity of snapshots
cat Print to stdout the specified file, or the snapshot content if no file is specified
diff Compare two snapshots or two revisions of a file
history Show the history of a file
prune Prune snapshots by revision, tag, or retention policy
password Change the storage password
add Add an additional storage to be used for the existing repository
set Change the options for the default or specified storage
copy Copy snapshots between compatible storages
info Show the information about the specified storage
benchmark Run a set of benchmarks to test download and upload speeds
help, h Shows a list of commands or help for one command

GLOBAL OPTIONS:
-verbose, -v show more detailed information
-debug, -d show even more detailed information, useful for debugging
-log enable log-style output
-stack print the stack trace when an error occurs
-no-script do not run script before or after command execution
-background read passwords, tokens, or keys only from keychain/keyring or env
-profile address:port enable the profiling tool and listen on the specified address:port
-comment add a comment to identify the process
-suppress, -s [+] suppress logs with the specified id
-print-memory-usage print memory usage every second
-help, -h show help

./duplicacy_linux_x64_3.2.3 -h 0.04s user 0.04s system 0% cpu 2:00.15 total

Please help, Thanks.

@realzhang realzhang changed the title duplicacy command startup is extreamly slow duplicacy command startup is extremely slow, does it do some network connection check? Oct 12, 2023
@sw684
Copy link

sw684 commented Jul 7, 2024

Experiencing this issue as well on Debian 12, takes exactly 2 minutes to start

@realzhang
Copy link
Author

any help?

@anonymouspage
Copy link

@realzhang @sw684 I created a Debian 12 VM and can't reproduce this, even with networking disabled. If you are still having the problem and don't mind a little work, run the following and attach the output:

strace -tttrf -o strace.log ./duplicacy_linux_x64_3.2.3 -h

If strace isn't installed, first install it: sudo apt install strace. A log file named strace.log will be created, you can attach it here and we can see what's going on. This will give us a little black box insight to see if the hang is due to a blocking call into the kernel.

When you are done, you can remove strace if you had to install it first: sudo apt remove strace

@realzhang
Copy link
Author

strace.log
duplicacy -h never ends with strace.log size keeping increase.

@anonymouspage
Copy link

anonymouspage commented Jan 20, 2025

@realzhang I should have also asked initially, but please comment with the output of uname -a and cat /etc/os-release and cat /proc/cpuinfo.

Also, please try running it with the following command (for testing purposes only, if this happens to work, performance would be very degraded, it's just a data point):
taskset --cpu-list 0 ./duplicacy_linux_x64_3.2.3 -h

@realzhang
Copy link
Author

The output of uname -a and cat /etc/os-release are pasted below. cpuinfo is attached as text file because of its length.
taskset --cpu-list 0 ./duplicacy_linux_x64_3.2.3 -h does not work either, but one difference, the strace_cpu0.log of strace -tttrf -o strace_cpu0.log taskset --cpu-list 0 ./duplicacy_linux_x64_3.2.3 -h is not increasing, which is attached too.

cpuinfo.txt

strace_cpu0.log

$ uname -a                       
Linux nserver 5.14.0-503.15.1.el9_5.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Nov 28 07:25:19 EST 2024 x86_64 x86_64 x86_64 GNU/Linux

$ cat /etc/os-release            
NAME="AlmaLinux"
VERSION="9.5 (Teal Serval)"
ID="almalinux"
ID_LIKE="rhel centos fedora"
VERSION_ID="9.5"
PLATFORM_ID="platform:el9"
PRETTY_NAME="AlmaLinux 9.5 (Teal Serval)"
ANSI_COLOR="0;34"
LOGO="fedora-logo-icon"
CPE_NAME="cpe:/o:almalinux:almalinux:9::baseos"
HOME_URL="https://almalinux.org/"
DOCUMENTATION_URL="https://wiki.almalinux.org/"
BUG_REPORT_URL="https://bugs.almalinux.org/"

ALMALINUX_MANTISBT_PROJECT="AlmaLinux-9"
ALMALINUX_MANTISBT_PROJECT_VERSION="9.5"
REDHAT_SUPPORT_PRODUCT="AlmaLinux"
REDHAT_SUPPORT_PRODUCT_VERSION="9.5"
SUPPORT_END=2032-06-01

@anonymouspage
Copy link

@realzhang Interesting. You have a very high core count. Unfortunately I don't have a 256 core machine laying around, but I'll see if qemu will let me pretend. My guess is it will be so slow I wouldn't know if I was recreating the problem or not. :)

When you ran taskset --cpu-list 0 ./duplicacy_linux_x64_3.2.3 -h did the program ever exit on its own, or did you have to interrupt it? The trace ends rather abruptly and I don't see any sign that you interrupted it.

@realzhang
Copy link
Author

I hit Ctrl+C to interrupt taskset --cpu-list 0 ./duplicacy_linux_x64_3.2.3 -h, as it was stuck and the strace_cpu0.log file seemed repeating its last 9 lines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants