Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Array.Parallel.groupBy added #14895

Merged
merged 5 commits into from
Mar 15, 2023
Merged

Conversation

T-Gro
Copy link
Member

@T-Gro T-Gro commented Mar 14, 2023

This is next part of the RFC to expand the Array.Parallel module, based on fsharp/fslang-suggestions#187

GroupBy Benchmarks were used for picking an appropriate implementation, comparing against:
Sequential Array.groupBy
PLINQ

The tricky part was balancing the optimization with regards to varying settings. Beyond others:
Struct vs Ref
Expensive vs Cheap projection function
Cardinality of the results (# of buckets)

Some properties desired about the results:

  • Invokes projection exactly once per input
  • Has predictable allocation profile, allocating typically a lot less then PLINQ
  • In most of the cases, at least 2x faster than LINQ (for some reasonable configurations, the speed over PLINQ can be up to 5x-7x better)
  • If it is slower then PLINQ, it is so only in rarer cases

One of the results contains measurement for the full configuration matrix:
https://github.com/T-Gro/Fsharp.ParallelGroupBy.Benchmarks/blob/main/Results/Detailed_Against_PLINQ_Full_Config_Matrix.md

@T-Gro T-Gro marked this pull request as ready for review March 14, 2023 17:36
@T-Gro T-Gro requested a review from a team as a code owner March 14, 2023 17:36
@T-Gro T-Gro merged commit 1998f8d into dotnet:main Mar 15, 2023
kant2002 pushed a commit to kant2002/fsharp that referenced this pull request Apr 1, 2023
* Array.Parallel.groupBy added
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants