Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Benchmark / diffview against CRAN release v1.10.0 #868

Draft
wants to merge 291 commits into
base: v1.10.1
Choose a base branch
from
Draft

Conversation

lorenzwalthert
Copy link
Collaborator

@lorenzwalthert lorenzwalthert commented Nov 28, 2021

DO NOT MERGE, this is for benchmarking only.

You may want to unsubscribe from this PR notifications, as any merge into main will trigger notifications.


Comparision with previous releases:

Releae 1.7.0

  •   :rocket:cache_applying: 29.1ms -> 28.2ms [-5.98%, -0.35%]
  •   :rocket:cache_recording: 1.24s -> 1.22s [-1.91%, -0.46%]
  • ❗🐌without_cache: 3.21s -> 3.3s [+2.1%, +3.94%]

@lorenzwalthert lorenzwalthert marked this pull request as draft November 28, 2021 18:41
@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 7c0acd7 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 34.2ms -> 34.5ms [-2.41%, +4.22%]
  •   :rocket:cache_recording: 1.64s -> 1.6s [-3.17%, -0.39%]
  •   :ballot_box_with_check:without_cache: 4.24s -> 4.21s [-2.01%, +0.28%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@codecov-commenter
Copy link

codecov-commenter commented Nov 28, 2021

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.07%. Comparing base (8f839e6) to head (8208348).

Current head 8208348 differs from pull request most recent head 7b5327c

Please upload reports for the commit 7b5327c to get more accurate results.

Additional details and impacted files
@@             Coverage Diff             @@
##           v1.10.1     #868      +/-   ##
===========================================
- Coverage    92.33%   91.07%   -1.27%     
===========================================
  Files           46       46              
  Lines         2650     2711      +61     
===========================================
+ Hits          2447     2469      +22     
- Misses         203      242      +39     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3c14e96 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 24.5ms -> 24.6ms [-2.03%, +3.34%]
  •   :rocket:cache_recording: 1.07s -> 1.02s [-6.4%, -1.18%]
  •   :ballot_box_with_check:without_cache: 2.76s -> 2.71s [-4.17%, +0.67%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 6b8ad76 is merged into v1.6.2:

  •   :rocket:cache_applying: 30.4ms -> 29.3ms [-6.12%, -0.8%]
  •   :rocket:cache_recording: 1.07s -> 1.03s [-4.73%, -2.87%]
  •   :ballot_box_with_check:without_cache: 2.83s -> 2.8s [-2.15%, +0.09%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 8, 2021

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fcd4c19 is merged into v1.6.2:

  •   :rocket:cache_applying: 30.3ms -> 29.1ms [-7.04%, -0.98%]
  •   :rocket:cache_recording: 1.1s -> 1.06s [-3.82%, -2.84%]
  •   :rocket:without_cache: 2.83s -> 2.8s [-1.52%, -0.36%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3ef57c2 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 29.8ms -> 30.5ms [-8.12%, +12.32%]
  •   :rocket:cache_recording: 1.08s -> 1.04s [-3.79%, -3.05%]
  •   :rocket:without_cache: 2.82s -> 2.79s [-1.28%, -0.73%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ea7456d is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 34.3ms -> 34.3ms [-3.76%, +3.95%]
  •   :rocket:cache_recording: 1.42s -> 1.38s [-4.29%, -1.6%]
  •   :ballot_box_with_check:without_cache: 3.74s -> 3.73s [-1.38%, +1.12%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if deff5e9 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 35.6ms -> 34.2ms [-8.58%, +0.94%]
  •   :rocket:cache_recording: 1.44s -> 1.39s [-5.06%, -2.4%]
  •   :ballot_box_with_check:without_cache: 3.78s -> 3.75s [-1.8%, +0.41%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b8f7359 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 31.9ms -> 32.1ms [-0.45%, +1.41%]
  •   :rocket:cache_recording: 1.18s -> 1.15s [-3.32%, -2.09%]
  •   :ballot_box_with_check:without_cache: 3.11s -> 3.09s [-0.83%, 0%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 988a9ac is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 36.1ms -> 35.3ms [-5.96%, +1.58%]
  •   :ballot_box_with_check:cache_recording: 1.5s -> 1.47s [-5.48%, +0.67%]
  •   :ballot_box_with_check:without_cache: 3.88s -> 3.88s [-1.7%, +1.2%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4fd6aee is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 29.2ms -> 29.3ms [-0.33%, +1.46%]
  •   :rocket:cache_recording: 1.06s -> 1.03s [-3.41%, -2.38%]
  •   :ballot_box_with_check:without_cache: 2.75s -> 2.75s [-0.83%, +0.36%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d0e4f92 is merged into v1.6.2:

  •   :rocket:cache_applying: 32.1ms -> 29ms [-11.97%, -7.08%]
  •   :rocket:cache_recording: 1.17s -> 1.15s [-2.22%, -1%]
  •   :ballot_box_with_check:without_cache: 3.04s -> 3.05s [-0.14%, +0.75%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if c1a46e4 is merged into v1.6.2:

  •   :rocket:cache_applying: 36.2ms -> 33.4ms [-10.71%, -4.65%]
  •   :rocket:cache_recording: 1.5s -> 1.47s [-2.75%, -1.35%]
  •   :ballot_box_with_check:without_cache: 3.94s -> 3.96s [-0.23%, +1.31%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ef4216a is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 38.7ms -> 38.8ms [-1.95%, +2.75%]
  •   :rocket:cache_recording: 1.62s -> 1.58s [-3.42%, -1.31%]
  •   :ballot_box_with_check:without_cache: 4.25s -> 4.25s [-1.07%, +0.94%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 63a874c is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.5ms -> 27.4ms [-1.14%, +0.42%]
  •   :rocket:cache_recording: 1.01s -> 989ms [-2.75%, -1.9%]
  •   :ballot_box_with_check:without_cache: 2.68s -> 2.69s [-0.43%, +0.74%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 9a169bb is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 28ms -> 28ms [-0.99%, +0.38%]
  •   :rocket:cache_recording: 1.04s -> 1.01s [-2.92%, -2.07%]
  •   :ballot_box_with_check:without_cache: 2.69s -> 2.7s [-0.02%, +0.99%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if c6fc0e3 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 30.4ms -> 30.3ms [-1.59%, +0.72%]
  •   :rocket:cache_recording: 1.1s -> 1.07s [-3.43%, -2.33%]
  • ❗🐌without_cache: 2.85s -> 2.87s [+0.16%, +1.11%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if fac168b is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.2ms -> 27ms [-1.03%, +0.09%]
  •   :rocket:cache_recording: 1s -> 976ms [-3.16%, -2.22%]
  • ❗🐌without_cache: 2.6s -> 2.62s [+0.2%, +1.05%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d338888 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 37.3ms -> 36.9ms [-3.53%, +1.46%]
  •   :ballot_box_with_check:cache_recording: 1.54s -> 1.52s [-3.35%, +0.33%]
  •   :ballot_box_with_check:without_cache: 4.01s -> 4.03s [-0.3%, +1.56%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if d91d98c is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 28.3ms -> 28.3ms [-0.8%, +0.84%]
  •   :rocket:cache_recording: 1.02s -> 1s [-2.64%, -1.29%]
  • ❗🐌without_cache: 2.67s -> 2.69s [+0.04%, +1.23%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 5fcc5e5 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 31.3ms -> 31.2ms [-1.92%, +1.8%]
  •   :rocket:cache_recording: 1.1s -> 1.08s [-3.04%, -1.81%]
  •   :ballot_box_with_check:without_cache: 2.89s -> 2.88s [-0.96%, +0.43%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 14dcc65 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 32ms -> 32ms [-2.41%, +2.42%]
  •   :rocket:cache_recording: 1.14s -> 1.12s [-3.91%, -0.12%]
  •   :ballot_box_with_check:without_cache: 3.03s -> 3.04s [-1.19%, +2.3%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

github-actions bot commented Jan 3, 2022

This is how benchmark results would change (along with a 95% confidence interval in relative change) if b1902ec is merged into v1.6.2:

  •   :rocket:cache_applying: 30.4ms -> 26.7ms [-16.18%, -8.41%]
  •   :rocket:cache_recording: 1.09s -> 1.05s [-6.17%, -1.14%]
  •   :ballot_box_with_check:without_cache: 2.8s -> 2.82s [-1.87%, +3.18%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@lorenzwalthert lorenzwalthert mentioned this pull request Jan 15, 2022
@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 27761d1 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 27.5ms -> 27.4ms [-4.08%, +3.33%]
  •   :rocket:cache_recording: 1.29s -> 1.26s [-3.56%, -0.98%]
  •   :ballot_box_with_check:without_cache: 3.39s -> 3.38s [-1.45%, +1.22%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 8f76e8f is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 33ms -> 33.6ms [-3.66%, +6.99%]
  •   :rocket:cache_recording: 1.61s -> 1.56s [-4.67%, -1.12%]
  •   :ballot_box_with_check:without_cache: 4.25s -> 4.23s [-2.17%, +0.97%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 59cf703 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 32.6ms -> 31.8ms [-5.58%, +0.98%]
  •   :rocket:cache_recording: 1.55s -> 1.5s [-4.27%, -2.28%]
  •   :ballot_box_with_check:without_cache: 4.09s -> 4.09s [-0.92%, +0.6%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 6, 2022

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 3a92151 is merged into v1.6.2:

  •   :ballot_box_with_check:cache_applying: 25.2ms -> 24.7ms [-4.78%, +0.69%]
  •   :rocket:cache_recording: 1.09s -> 1.05s [-3.96%, -2.79%]
  •   :ballot_box_with_check:without_cache: 2.83s -> 2.83s [-0.75%, +0.41%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 59c3d34 is merged into v1.10.1:

  • ✔️cache_applying: 158ms -> 159ms [-7.92%, +9.52%]
  • ✔️cache_recording: 523ms -> 533ms [-1.76%, +5.39%]
  • ✔️without_cache: 994ms -> 1s [-1.66%, +2.95%]

Further explanation regarding interpretation and methodology can be found in the documentation.

dependabot bot and others added 2 commits December 2, 2024 03:22
Bumps [JamesIves/github-pages-deploy-action](https://github.com/jamesives/github-pages-deploy-action) from 4.6.9 to 4.7.1.
- [Release notes](https://github.com/jamesives/github-pages-deploy-action/releases)
- [Commits](JamesIves/github-pages-deploy-action@v4.6.9...v4.7.1)

---
updated-dependencies:
- dependency-name: JamesIves/github-pages-deploy-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Copy link
Contributor

github-actions bot commented Dec 2, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 73fd5a6 is merged into v1.10.1:

  • ❗🐌cache_applying: 165ms -> 171ms [+0.21%, +7.55%]
  • ✔️cache_recording: 539ms -> 546ms [-1.13%, +3.67%]
  • ❗🐌without_cache: 1s -> 1.01s [+0.43%, +1.41%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Copy link
Contributor

github-actions bot commented Dec 4, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 2f1cc9b is merged into v1.10.1:

  • ✔️cache_applying: 150ms -> 152ms [-2.01%, +5.05%]
  • ✔️cache_recording: 535ms -> 541ms [-0.32%, +2.43%]
  • ❗🐌without_cache: 972ms -> 982ms [+0.4%, +1.73%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Copy link
Contributor

github-actions bot commented Dec 8, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if c637dc0 is merged into v1.10.1:

  • ✔️cache_applying: 154ms -> 153ms [-2.38%, +2.3%]
  • ✔️cache_recording: 520ms -> 523ms [-0.89%, +2.02%]
  • ✔️without_cache: 986ms -> 989ms [-0.71%, +1.32%]

Further explanation regarding interpretation and methodology can be found in the documentation.

dependabot bot and others added 2 commits December 9, 2024 03:47
Bumps [JamesIves/github-pages-deploy-action](https://github.com/jamesives/github-pages-deploy-action) from 4.7.1 to 4.7.2.
- [Release notes](https://github.com/jamesives/github-pages-deploy-action/releases)
- [Commits](JamesIves/github-pages-deploy-action@v4.7.1...v4.7.2)

---
updated-dependencies:
- dependency-name: JamesIves/github-pages-deploy-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Copy link
Contributor

github-actions bot commented Dec 9, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4f1d0c0 is merged into v1.10.1:

  • ✔️cache_applying: 163ms -> 167ms [-1.39%, +6.88%]
  • ✔️cache_recording: 525ms -> 529ms [-0.77%, +2.22%]
  • ✔️without_cache: 1.01s -> 1.02s [-0.12%, +2.14%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Instead of `expect_warning()` + `NA` combo
Copy link
Contributor

github-actions bot commented Dec 9, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 4e3bc30 is merged into v1.10.1:

  • ✔️cache_applying: 178ms -> 178ms [-2.28%, +1.99%]
  • ✔️cache_recording: 541ms -> 548ms [-0.37%, +2.9%]
  • ❗🐌without_cache: 1.02s -> 1.03s [+0.55%, +1.99%]

Further explanation regarding interpretation and methodology can be found in the documentation.

We are being unnecessarily terse in some places where we don't need to be. I always feel like I am hitting a little speed bump every time I need to decode these names mentally.
Copy link
Contributor

github-actions bot commented Dec 9, 2024

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ba3fe63 is merged into v1.10.1:

  • ❗🐌cache_applying: 139ms -> 142ms [+1%, +3.42%]
  • ❗🐌cache_recording: 498ms -> 506ms [+0.67%, +2.47%]
  • ❗🐌without_cache: 958ms -> 968ms [+0.57%, +1.46%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 84f250a is merged into v1.10.1:

  • ✔️cache_applying: 154ms -> 158ms [-0.92%, +5.66%]
  • ❗🐌cache_recording: 518ms -> 526ms [+0.23%, +2.78%]
  • ✔️without_cache: 994ms -> 1s [-0.05%, +1.67%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants