Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks #285

Merged
merged 3 commits into from
Mar 29, 2024
Merged

Add benchmarks #285

merged 3 commits into from
Mar 29, 2024

Conversation

turion
Copy link
Owner

@turion turion commented Feb 5, 2024

  • Clean up git history
  • Is it legal to carry the Gutenberg file around?

@turion
Copy link
Owner Author

turion commented Mar 11, 2024

Possibly cherry pick adb464a?

@turion
Copy link
Owner Author

turion commented Mar 13, 2024

Is it legal to carry the Gutenberg file around?

I found another version of the file that is certainly legal to use here.

@turion
Copy link
Owner Author

turion commented Mar 13, 2024

The new benchmark data file has more lines, which makes it even slower:

benchmarking WordCount/rhine
time                 3.733 s    (3.474 s .. 3.992 s)
                     0.999 R²   (0.998 R² .. 1.000 R²)
mean                 3.738 s    (3.706 s .. 3.783 s)
std dev              43.07 ms   (15.49 ms .. 56.79 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/dunai
time                 999.1 ms   (912.9 ms .. 1.076 s)
                     0.999 R²   (0.999 R² .. 1.000 R²)
mean                 1.032 s    (1.013 s .. 1.046 s)
std dev              19.56 ms   (9.140 ms .. 27.15 ms)
variance introduced by outliers: 19% (moderately inflated)

benchmarking WordCount/Text/IORef
time                 81.32 ms   (80.70 ms .. 82.68 ms)
                     1.000 R²   (0.999 R² .. 1.000 R²)
mean                 80.79 ms   (80.39 ms .. 81.25 ms)
std dev              745.6 μs   (506.6 μs .. 1.029 ms)

benchmarking WordCount/Text/no IORef
time                 164.9 ms   (130.1 ms .. 213.0 ms)
                     0.952 R²   (0.847 R² .. 0.999 R²)
mean                 173.6 ms   (162.1 ms .. 188.7 ms)
std dev              18.36 ms   (12.95 ms .. 24.35 ms)
variance introduced by outliers: 27% (moderately inflated)

benchmarking WordCount/Text/Lazy
time                 100.6 ms   (92.54 ms .. 108.5 ms)
                     0.991 R²   (0.981 R² .. 1.000 R²)
mean                 100.6 ms   (98.10 ms .. 103.2 ms)
std dev              4.322 ms   (3.407 ms .. 6.058 ms)

It's clear that Rhine and dunai have an overhead per tick, since the benchmark programs operate line-wise.

rhine/bench/WordCount.hs Outdated Show resolved Hide resolved
@turion turion mentioned this pull request Mar 13, 2024
4 tasks
@turion turion force-pushed the dev_benchmark branch 5 times, most recently from 1707d45 to d0c686b Compare March 29, 2024 14:19
@turion turion enabled auto-merge March 29, 2024 14:19
@turion turion merged commit 8c9e74b into master Mar 29, 2024
13 checks passed
@turion turion deleted the dev_benchmark branch March 29, 2024 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant