Speed up Glances launch

[nicolargo]

Description

Glances is slow to start.

Make some optimization in order to speed up it.

Versions

• Glances & psutil (glances -V): Glances v3.1.0 with psutil v5.6.1
• Operating System (lsb_release -a): Ubuntu 18/04

Logs

N/A

[nicolargo]

Glances 3.1.2 beta: Glances started in 3.214372 seconds

Plugins start duration, total: 1.3 seconds (see detail below)
First update duration: 1.9 seconds (see detail below)

Plugins start duration, detail:
glances_sensors.py | 0.858416

Focus on sensor plugin:
2019-08-22 17:21:25,168 -- DEBUG -- Generic sensor plugin init duration: 0.514641 seconds
2019-08-22 17:21:25,168 -- DEBUG -- HDDTemp sensor plugin init duration: 0.000228 seconds
2019-08-22 17:21:25,404 -- DEBUG -- Battery sensor plugin init duration: 0.235736 seconds

glances_ip.py | 0.21727
glances_amps.py | 0.090728
glances_docker.py | 0.038769
glances_processlist.py | 0.021272
glances_quicklook.py | 0.020084
glances_wifi.py | 0.010152
glances_folders.py | 0.009877
glances_smart.py | 0.009162
glances_ports.py | 0.006738
glances_gpu.py | 0.005562
glances_percpu.py | 0.002726
glances_raid.py | 0.002264
glances_cpu.py | 0.001849
glances_load.py | 0.001554
glances_processcount.py | 0.001525
glances_cloud.py | 0.001451
glances_diskio.py | 0.001427
glances_memswap.py | 0.001285
glances_fs.py | 0.001167
glances_network.py | 0.000968
glances_help.py | 0.000943
glances_system.py | 0.000769
glances_mem.py | 0.00076
glances_psutilversion.py | 0.000639
glances_alert.py | 0.000488
glances_irq.py | 0.00045
glances_uptime.py | 0.000421
glances_now.py | 0.000329
glances_core.py | 0.00029

Plugin | Update duration
sensors | 1.255504
folders | 0.511612
processcount | 0.12157
quicklook | 0.034188
amps | 0.028799
wifi | 0.014103
docker | 0.012876
system | 0.005369
ports | 0.001083
fs | 0.000974
cpu | 0.000751
diskio | 0.000583
irq | 0.000515
percpu | 0.000456
ip | 0.000455
mem | 0.000359
network | 0.000348
memswap | 0.000336
core | 0.000318
raid | 0.000285
load | 0.000282
uptime | 0.000165
now | 0.000141
gpu | 0.000104
psutilversion | 0.000103
cloud | 6.70E-05
processlist | 4.10E-05
alert | 1.80E-05
help | 1.10E-05

[nicolargo]

Sensors and folders are good candidates for optimization...

[nicolargo]

Run Glances in debug mode (-d) and grep "duration" in the Glances log file.

[God-damnit-all]

Not sure if Windows is at all a priority, but it is much, much slower to start up in Windows, and for some reason there's quite a delay when using keyboard shortcuts after it starts up. It took about 15 seconds to start up and about 6 seconds for it to register the 'q' for 'quit' command.

[nicolargo]

@ImportTaste i think that there is an issue with one of the plugin. Can you please try to run Glances with the --disable-plugin sensors option.

You can run and copy/paste the result of:

glances --issue

Thanks

[God-damnit-all]

@ImportTaste i think that there is an issue with one of the plugin. Can you please try to run Glances with the --disable-plugin sensors option.

You can run and copy/paste the result of:

glances --issue

Thanks

I emailed you the results using the contact email on your profile.

[nicolargo]

Two plugins in your email take lot of time to process:

• processcount (this plugin grabs all the processes stats): ~ 12 seconds
• sensors: ~ 2 seconds

On my system, it takes:

• processcount: 0.07 second
• sensors: 0,001 second

Glances uses the PsUtil lib to get the processes and sensors stats. Something is wrong on your system with this lib. In order to confirm the behavor, can you run this command line on your system:

python -m timeit -c "import psutil; procs = {p.pid: p for p in psutil.process_iter()};"

Result on my PC: 50 loops, best of 5: 8.77 msec per loop

and this other one:

python -m timeit -c "import psutil; procs = {p.pid: p for p in psutil.process_iter(attrs=['cpu_percent', 'cpu_times', 'memory_percent', 'name', 'status', 'num_threads', 'io_counters'])};"

Result on my PC: 50 loops, best of 5: 55.02 msec per loop

[God-damnit-all]

Yeah, I get: "1 loop, best of 5: 2.08 sec per loop" Even when I use Python 3.8 instead of 3.10, it comes out to about the same. I guess this lib's use case is just very slow on Windows.

[nicolargo]

@ImportTaste Try to open an issue with the result of the previous command line on the PsUtil Github repository: https://github.com/giampaolo/psutil/issues

Perhaps related to giampaolo/psutil#1967 ?

[nicolargo]

With Glances 4.0.0 (beta version may 2023):

2023-05-18 19:07:06,737 -- DEBUG -- Generic sensor plugin init duration: 0.010017 seconds
2023-05-18 19:07:06,739 -- DEBUG -- HDDTemp sensor plugin init duration: 0.001671 seconds
2023-05-18 19:07:06,742 -- DEBUG -- Battery sensor plugin init duration: 0.002811 seconds
2023-05-18 19:07:06,749 -- DEBUG -- Plugins initialisation duration: 0.32285 seconds
2023-05-18 19:07:07,795 -- DEBUG -- First stats update duration: 1.045783 seconds
2023-05-18 19:07:07,813 -- DEBUG -- Stats updated duration: 0.015454 seconds
...

and without the container plugin:

2023-05-18 19:09:49,620 -- DEBUG -- Generic sensor plugin init duration: 0.009507 seconds
2023-05-18 19:09:49,621 -- DEBUG -- HDDTemp sensor plugin init duration: 0.001502 seconds
2023-05-18 19:09:49,624 -- DEBUG -- Battery sensor plugin init duration: 0.00276 seconds
2023-05-18 19:09:49,631 -- DEBUG -- Plugins initialisation duration: 0.432124 seconds
2023-05-18 19:09:50,557 -- DEBUG -- First stats update duration: 0.926544 seconds
2023-05-18 19:09:50,574 -- DEBUG -- Stats updated duration: 0.014492 seconds
...

[God-damnit-all]

That's quite the improvement, what turned out to be the main issue?

And is the beta available anywhere, or is currently internal only?

[KaiStarkk]

@nicolargo a large part of the duration for starting up the webUI seems to be loading the favicon.

"GET /87708faeed9a66b0fcdb.png HTTP/1.1" 200 43588

[nicolargo]

@KaiStarkk the favicon is not downloaded from Internet but provided in the Glances package. Perhaps the issue came from your ad-blocker because it did not allow HTTP local trafic to the Favicon URL.

[KaiStarkk]

It doesn't seem to be an adblock issue, same issue even with extensions disabled. API is working fine, issue only occurs when trying to load the WebUI directly. While trying to load the WebUI though, API requests get refused for about 10 seconds. Not a major issue as I'm only using the API, but yeah strange.

…

Message ID: ***@***.***>

[nicolargo]

Strange, can not reproduce on my side... What's your Glances version / Operating system ?

[KaiStarkk]

Glances v3.4.0.3 with PsUtil v5.9.5

Systeminfo:

OS Name:                   Microsoft Windows 10 Pro
OS Version:                10.0.19044 N/A Build 19044
System Model:              MS-7C02
System Type:               x64-based PC
Hotfix(s):                 8 Hotfix(s) Installed.
                           [01]: KB5018329
                           [02]: KB5003791
                           [03]: KB5018410
                           [04]: KB5007273
                           [05]: KB5009636
                           [06]: KB5011352
                           [07]: KB5014032
                           [08]: KB5016705
Hyper-V Requirements:      A hypervisor has been detected. Features required for Hyper-V will not be displayed.

Let me know if any logging would be helpful. It's not a critical bug for me so no stress if it's low priority.

[nicolargo]

Glances 4.0.0 (beta):

Minimal (Glances started in 0.525292 seconds).

tail -f /home/nicolargo/.local/share/glances/glances.log | grep started
2024-03-16 10:06:54,299 -- DEBUG -- Plugin raid started in 0.004368 seconds
2024-03-16 10:06:54,300 -- DEBUG -- Plugin core started in 0.001083 seconds
2024-03-16 10:06:54,305 -- DEBUG -- Plugin now started in 0.003894 seconds
2024-03-16 10:06:54,307 -- DEBUG -- Plugin fs started in 0.001902 seconds
2024-03-16 10:06:54,308 -- DEBUG -- Plugin memswap started in 0.001452 seconds
2024-03-16 10:06:54,314 -- DEBUG -- Plugin processlist started in 0.005602 seconds
2024-03-16 10:06:54,315 -- DEBUG -- Plugin system started in 0.001324 seconds
2024-03-16 10:06:54,340 -- DEBUG -- Plugin cpu started in 0.023906 seconds
2024-03-16 10:06:54,349 -- DEBUG -- Plugin amps started in 0.009441 seconds
2024-03-16 10:06:54,352 -- DEBUG -- Plugin cloud started in 0.002977 seconds
2024-03-16 10:06:54,354 -- DEBUG -- Plugin help started in 0.001421 seconds
2024-03-16 10:06:54,357 -- DEBUG -- Plugin load started in 0.003121 seconds
2024-03-16 10:06:54,360 -- DEBUG -- Plugin smart started in 0.002127 seconds
2024-03-16 10:06:54,363 -- DEBUG -- Plugin folders started in 0.002803 seconds
2024-03-16 10:06:54,376 -- DEBUG -- Plugin ports started in 0.013092 seconds
2024-03-16 10:06:54,383 -- DEBUG -- Plugin quicklook started in 0.006594 seconds
2024-03-16 10:06:54,389 -- DEBUG -- Plugin alert started in 0.006222 seconds
2024-03-16 10:06:54,392 -- DEBUG -- Plugin uptime started in 0.001848 seconds
2024-03-16 10:06:54,396 -- DEBUG -- Plugin percpu started in 0.003869 seconds
2024-03-16 10:06:54,402 -- DEBUG -- Plugin network started in 0.005723 seconds
2024-03-16 10:06:54,405 -- DEBUG -- Plugin mem started in 0.002588 seconds
2024-03-16 10:06:54,408 -- DEBUG -- Plugin ip started in 0.003364 seconds
2024-03-16 10:06:54,413 -- DEBUG -- Plugin gpu started in 0.004043 seconds
2024-03-16 10:06:54,415 -- DEBUG -- Plugin processcount started in 0.001917 seconds
2024-03-16 10:06:54,417 -- DEBUG -- Plugin version started in 0.001449 seconds
2024-03-16 10:06:54,423 -- DEBUG -- Plugin containers started in 0.005773 seconds
2024-03-16 10:06:54,425 -- DEBUG -- Plugin connections started in 0.002457 seconds
2024-03-16 10:06:54,429 -- DEBUG -- Plugin wifi started in 0.00293 seconds
2024-03-16 10:06:54,431 -- DEBUG -- Plugin diskio started in 0.002459 seconds
2024-03-16 10:06:54,481 -- DEBUG -- Plugin sensors started in 0.049086 seconds
2024-03-16 10:06:54,483 -- DEBUG -- Plugin irq started in 0.001813 seconds
2024-03-16 10:06:54,484 -- DEBUG -- Plugin psutilversion started in 0.001264 seconds
2024-03-16 10:06:54,485 -- DEBUG -- Exporter graph started in 1.6e-05 seconds
2024-03-16 10:06:54,486 -- DEBUG -- Exporter csv started in 1.6e-05 seconds
2024-03-16 10:06:54,486 -- DEBUG -- Exporter json started in 1.5e-05 seconds
2024-03-16 10:06:54,486 -- DEBUG -- Exporter rabbitmq started in 1.5e-05 seconds
2024-03-16 10:06:54,486 -- DEBUG -- Exporter riemann started in 1.5e-05 seconds
2024-03-16 10:06:54,487 -- DEBUG -- Exporter prometheus started in 2e-05 seconds
2024-03-16 10:06:54,487 -- DEBUG -- Exporter kafka started in 1.8e-05 seconds
2024-03-16 10:06:54,488 -- DEBUG -- Exporter graphite started in 1.5e-05 seconds
2024-03-16 10:06:54,488 -- DEBUG -- Exporter elasticsearch started in 1.5e-05 seconds
2024-03-16 10:06:54,488 -- DEBUG -- Exporter zeromq started in 1.6e-05 seconds
2024-03-16 10:06:54,489 -- DEBUG -- Exporter couchdb started in 1.8e-05 seconds
2024-03-16 10:06:54,489 -- DEBUG -- Exporter influxdb started in 2e-05 seconds
2024-03-16 10:06:54,489 -- DEBUG -- Exporter mongodb started in 1.6e-05 seconds
2024-03-16 10:06:54,490 -- DEBUG -- Exporter mqtt started in 1.6e-05 seconds
2024-03-16 10:06:54,490 -- DEBUG -- Exporter cassandra started in 1.6e-05 seconds
2024-03-16 10:06:54,490 -- DEBUG -- Exporter restful started in 1.4e-05 seconds
2024-03-16 10:06:54,491 -- DEBUG -- Exporter statsd started in 2e-05 seconds
2024-03-16 10:06:54,491 -- DEBUG -- Exporter opentsdb started in 1.7e-05 seconds
2024-03-16 10:06:54,491 -- DEBUG -- Exporter influxdb2 started in 1.6e-05 seconds
2024-03-16 10:06:54,798 -- DEBUG -- Glances started in 0.525292 seconds

Full plugins (Glances started in 0.965322 seconds):

tail -f /home/nicolargo/.local/share/glances/glances.log | grep started
2024-03-16 10:09:05,509 -- DEBUG -- Plugin raid started in 0.005733 seconds
2024-03-16 10:09:05,510 -- DEBUG -- Plugin core started in 0.000849 seconds
2024-03-16 10:09:05,512 -- DEBUG -- Plugin now started in 0.002371 seconds
2024-03-16 10:09:05,514 -- DEBUG -- Plugin fs started in 0.001684 seconds
2024-03-16 10:09:05,516 -- DEBUG -- Plugin memswap started in 0.001532 seconds
2024-03-16 10:09:05,521 -- DEBUG -- Plugin processlist started in 0.004816 seconds
2024-03-16 10:09:05,522 -- DEBUG -- Plugin system started in 0.001238 seconds
2024-03-16 10:09:05,547 -- DEBUG -- Plugin cpu started in 0.025018 seconds
2024-03-16 10:09:05,649 -- DEBUG -- Plugin amps started in 0.101456 seconds
2024-03-16 10:09:05,651 -- DEBUG -- Plugin cloud started in 0.00254 seconds
2024-03-16 10:09:05,657 -- DEBUG -- Plugin help started in 0.004824 seconds
2024-03-16 10:09:05,661 -- DEBUG -- Plugin load started in 0.003775 seconds
2024-03-16 10:09:05,668 -- DEBUG -- Plugin smart started in 0.006423 seconds
2024-03-16 10:09:05,669 -- DEBUG -- Plugin folders started in 0.001444 seconds
2024-03-16 10:09:05,676 -- DEBUG -- Plugin ports started in 0.007144 seconds
2024-03-16 10:09:05,684 -- DEBUG -- Plugin quicklook started in 0.007001 seconds
2024-03-16 10:09:05,688 -- DEBUG -- Plugin alert started in 0.004005 seconds
2024-03-16 10:09:05,689 -- DEBUG -- Plugin uptime started in 0.000929 seconds
2024-03-16 10:09:05,691 -- DEBUG -- Plugin percpu started in 0.002157 seconds
2024-03-16 10:09:05,695 -- DEBUG -- Plugin network started in 0.003429 seconds
2024-03-16 10:09:05,696 -- DEBUG -- Plugin mem started in 0.001529 seconds
2024-03-16 10:09:05,698 -- DEBUG -- Plugin ip started in 0.001616 seconds
2024-03-16 10:09:05,709 -- DEBUG -- Plugin gpu started in 0.010609 seconds
2024-03-16 10:09:05,711 -- DEBUG -- Plugin processcount started in 0.001844 seconds
2024-03-16 10:09:05,712 -- DEBUG -- Plugin version started in 0.000841 seconds
2024-03-16 10:09:05,860 -- DEBUG -- Plugin containers started in 0.14832 seconds
2024-03-16 10:09:05,862 -- DEBUG -- Plugin connections started in 0.001367 seconds
2024-03-16 10:09:05,863 -- DEBUG -- Plugin wifi started in 0.001522 seconds
2024-03-16 10:09:05,865 -- DEBUG -- Plugin diskio started in 0.001225 seconds
2024-03-16 10:09:05,890 -- DEBUG -- Plugin sensors started in 0.025498 seconds
2024-03-16 10:09:05,892 -- DEBUG -- Plugin irq started in 0.001532 seconds
2024-03-16 10:09:05,894 -- DEBUG -- Plugin psutilversion started in 0.00115 seconds
2024-03-16 10:09:05,894 -- DEBUG -- Exporter graph started in 1.1e-05 seconds
2024-03-16 10:09:05,894 -- DEBUG -- Exporter csv started in 1e-05 seconds
2024-03-16 10:09:05,894 -- DEBUG -- Exporter json started in 1e-05 seconds
2024-03-16 10:09:05,894 -- DEBUG -- Exporter rabbitmq started in 1.1e-05 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter riemann started in 1e-05 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter prometheus started in 9e-06 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter kafka started in 9e-06 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter graphite started in 9e-06 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter elasticsearch started in 9e-06 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter zeromq started in 9e-06 seconds
2024-03-16 10:09:05,895 -- DEBUG -- Exporter couchdb started in 1.1e-05 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter influxdb started in 1e-05 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter mongodb started in 9e-06 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter mqtt started in 9e-06 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter cassandra started in 9e-06 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter restful started in 8e-06 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter statsd started in 1.1e-05 seconds
2024-03-16 10:09:05,896 -- DEBUG -- Exporter opentsdb started in 1e-05 seconds
2024-03-16 10:09:05,897 -- DEBUG -- Exporter influxdb2 started in 9e-06 seconds
2024-03-16 10:09:06,448 -- DEBUG -- Glances started in 0.965322 seconds