WebUI latency on Openshift with more than 1000-1500 packages

[unplugguitar]

Describe the bug

Running the latest Verdaccio over openshift and publishing more then 1500 packages makes the WebUI very slow. The loading time for the main page takes much time (it seems as it is loading all of the packages metadata on opening the page). Moving between pages and searching is also very slow.

To Reproduce
Steps to reproduce the behavior:

1. Upload a Verdaccio instance on Openshift 3.x (Make sure to bind the storage to persistant volume)
2. Upload at least 1500 different packages
3. Open the main page and feel the latency
4. You can also try to search packages and move between pages to feel the latency

Expected behavior

Docker || Kubernetes (please complete the following information):

• Docker verdaccio tag: 4.0.4
• Docker commands docker pull verdaccio/verdaccio:4.0.4
• Docker Version: Openshift v3.x

Configuration File (cat ~/.config/verdaccio/config.yaml)
default configuration with no uplinks

Environment information

*The data is in a private network and could not be provided.
The Verdaccio version is 4.0.4, no uplinks, no other special configurations

Debugging output

• $ NODE_DEBUG=request verdaccio display request calls (verdaccio <--> uplinks)
• $ DEBUG=express:* verdaccio enable extreme verdaccio debug mode (verdaccio api)
• $ npm -ddd prints:
• $ npm config get registry prints:

*The data is in a private network and could not be provided.

Additional context

[juanpicado]

The project is not ready for such amount of packages. That’s our goal. But we are not there yet.

Related #706

[unplugguitar]

Is adding support for that amount of packages is scheduled for the next release? I think it could be usefull as Verdaccio became very popular in projects that need these amounts of packages.

[juanpicado]

I don't see this shipped in the next releases. I agree this is useful, but, I'm betting for stability now due the limited resources.

This requires rethink couple of things

• Add support for big amount of packages eg: (Issue 706 cached packages ui#35) but the approach purposed does not move me, I'm not big fan of infinite scrolls.
• Default plugins storage never was tested with such amount of packages.
• It might requires new web endpoints (eg: pagination)
• It has to be reliable, thus, ship something that actually works well.
• All the things above must to be tested, automated, documented, configurable etc.

I see too much work on this, unless someone is willing to put on the table resources to achieve it, then, I'll be here for provide guidance.

[dviryamin]

As i can tell the problem is that when opening the home page the ui renders all the packages in the registry.
For quick solution i think that adding flag that disable home page(In such big registry it doesn't give value) and replace it with static welcome page will do the job.
I checked and minimum of 3-letters for search make the result small enough to be rendered for 10k registry.
For larger registries I support @juanpicado suggest to implement pagination capabilities and make benchmarks for storage plugins.
With these improvement verdaccio can be complete private repository and not only proxy solution.

I can start work on the quick solution immediately.

[juanpicado]

@dviryamin I like the idea, but, before you start, let me share some internal notes. We plan to add a new set of properties for UI to disable features, eg: disable long listing or search. So, your idea is really nice, like a light version of the front-end. Let's have a chat http://chat.verdaccio.org (along the week, I'm bit busy these days) about this perhaps we can coordinate better.

[juanpicado]

fyi [email protected] has an amazing performance improvement on scrolling. kudos to @priscilawebdev 👏