Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow limiting the number of rayon threads #12625

Open
cuviper opened this issue Jan 21, 2025 · 0 comments
Open

Allow limiting the number of rayon threads #12625

cuviper opened this issue Jan 21, 2025 · 0 comments
Labels
C-enhancement Category: Improvements

Comments

@cuviper
Copy link

cuviper commented Jan 21, 2025

Ref: rayon-rs/rayon#1227
cc @LelsersLasers

They are running Helix on a server with a high number of CPUs (208), but a relatively low number of processes (and threads) allowed per user -- ulimit -u is 512. The picker's rayon pool failed to start with ErrorKind::WouldBlock (EAGAIN).

Since nucleo explicitly sets num_threads for the pool, the general RAYON_NUM_THREADS environment variable doesn't work. I suggested using CPU affinity to limit std::thread::available_parallelism() instead, but that's not a great long-term solution to be forcing the use of particular CPUs for the entire process.

I see that Picker::new always passes None to get the default thread count, but I think it would make sense to have some configuration or similar environment variable so the user can override this.

@cuviper cuviper added the C-enhancement Category: Improvements label Jan 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-enhancement Category: Improvements
Projects
None yet
Development

No branches or pull requests

1 participant