Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support different invocations for different benchmarks #17

Open
caizixian opened this issue Jul 20, 2021 · 2 comments
Open

Support different invocations for different benchmarks #17

caizixian opened this issue Jul 20, 2021 · 2 comments
Labels
enhancement New feature or request

Comments

@caizixian
Copy link
Member

Also see #14
@caizixian
Copy link
Member Author

Steve:

This depends a lot on the behavior of the JIT, but also may change with concurrent GC

I would use our current default to start with

My preference would be to evolve the scripts to tune the number of invocations to a statistical target

DaCapo has something like that built in w.r.t. iterations, not invocations
Possibly it would work to use that flag on DaCapo and then you could get away with maybe only 10 invocations

But a key message is that different benchmarks behave very differently, so it is quite inefficient to apply the same rule to all

Since logs can be combined, if the baseline gave reasonable results, then someone who wanted "super" results would just run the whole thing N times, until the results were suitably clean

@caizixian caizixian mentioned this issue Jul 20, 2021
Open
@caizixian caizixian added the enhancement New feature or request label Jul 20, 2021
caizixian pushed a commit that referenced this issue Oct 26, 2021
@dependabot
Bump autopep8 from 1.5.7 to 1.6.0 (#17)
58b02de 
Bumps [autopep8](https://github.com/hhatto/autopep8) from 1.5.7 to 1.6.0.
- [Release notes](https://github.com/hhatto/autopep8/releases)
- [Commits](hhatto/autopep8@v1.5.7...v1.6.0)

---
updated-dependencies:
- dependency-name: autopep8
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
@caizixian
Copy link
Member Author

Note that plotty currently only supports filtering on a fix value rather than extrema.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@caizixian