Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize random starts with multithreading #56

Closed
wants to merge 1 commit into from

Conversation

alyst
Copy link
Contributor

@alyst alyst commented Mar 30, 2024

The PR adds use_threads= option to the rotate() method (off by default).
If enabled, it would use Threads.foreach() to parallelize the optimization across the random starts.

rotate() heavily relies on SVD, but it looks like the default LAPACK library on my system (OpenBLAS?) does not take advantage of multi-threading.
Might be different for the other BLAS libraries, but use_threads would provide an alternative way to utilize multi-threading.

Also note that this PR is based on #55 branch anticipating it would be merged first.
But I can also rebase it to the main branch depending on what the package author will decide.

Copy link

codecov bot commented Apr 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.77%. Comparing base (9f687b9) to head (cc8df8e).
Report is 8 commits behind head on main.

❗ Current head cc8df8e differs from pull request most recent head 1739f03. Consider uploading reports for the commit 1739f03 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #56      +/-   ##
==========================================
- Coverage   98.69%   91.77%   -6.92%     
==========================================
  Files          23       23              
  Lines         459      462       +3     
==========================================
- Hits          453      424      -29     
- Misses          6       38      +32     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@alyst alyst force-pushed the parallel_starts branch from cc8df8e to f2a1235 Compare April 25, 2024 15:56
@alyst alyst force-pushed the parallel_starts branch from f2a1235 to 1739f03 Compare April 30, 2024 16:06
Copy link
Contributor

Benchmark Results

main 1739f03... main/1739f033b005d3...
criterion_and_gradient!/FactorRotations.Biquartimin{FactorRotations.Oblique}() 0.553 ± 0.32 μs 0.567 ± 0.32 μs 0.975
criterion_and_gradient!/FactorRotations.CrawfordFerguson{FactorRotations.Oblique, Float64}(0.5) 0.389 ± 0.0051 μs 0.397 ± 0.019 μs 0.98
criterion_and_gradient!/FactorRotations.Geomin{Float64}(0.01) 0.919 ± 0.18 μs 0.929 ± 0.17 μs 0.989
criterion_and_gradient!/FactorRotations.Infomax{FactorRotations.Oblique}() 1.4 ± 0.5 μs 1.4 ± 0.5 μs 1
criterion_and_gradient!/FactorRotations.MinimumEntropy() 0.46 ± 0.17 μs 0.466 ± 0.17 μs 0.987
criterion_and_gradient!/FactorRotations.MinimumEntropyRatio() 14.3 μs 16.2 μs 0.882
criterion_and_gradient!/FactorRotations.Oblimax{FactorRotations.Oblique}() 0.314 ± 0.001 μs 0.315 ± 0.001 μs 0.997
criterion_and_gradient!/FactorRotations.Oblimin{FactorRotations.Oblique, Float64}(0.5) 0.397 ± 0.0081 μs 0.398 ± 0.009 μs 0.997
criterion_and_gradient!/FactorRotations.Oblimin{FactorRotations.Orthogonal, Float64}(0.5) 0.393 ± 0.008 μs 0.397 ± 0.01 μs 0.99
criterion_and_gradient!/FactorRotations.Quartimax() 0.218 ± 0.001 μs 0.217 ± 0.001 μs 1
criterion_and_gradient!/FactorRotations.Simplimax(5) 0.567 ± 0.029 μs 0.57 ± 0.031 μs 0.995
criterion_and_gradient!/FactorRotations.TandemCriterionI() 3.17 ± 0.3 μs 3.15 ± 0.3 μs 1.01
criterion_and_gradient!/FactorRotations.TandemCriterionII() 1.62 ± 0.079 μs 1.6 ± 1.5 μs 1.01
criterion_and_gradient!/FactorRotations.Varimax() 0.152 ± 0.0001 μs 0.157 ± 0.001 μs 0.968
time_to_load 0.518 ± 0.007 s 0.522 ± 0.002 s 0.992

@p-gw
Copy link
Owner

p-gw commented Jul 10, 2024

Sorry I just got around to commenting on this PR. I'm closing this in favour of #70 since the Folds.map implementation leads to similar performance, but somewhat more readable and short code. However it requires a breaking change since we cannot return ConvergenceError and must keep the RotationStates for logging purposes. I will release the changes in #70 as 0.4 soon.

@p-gw p-gw closed this Jul 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants