-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve CSV reporting of benchmarks, add stencil system benchmark #221
Conversation
cd72912
to
8329cc2
Compare
Check-perf-impact results: (e3a4351784531e1861bff0cb14c92210)
Overall relative execution time: 1.01x (mean of relative medians) |
This goes to show that the threshold of "significance" also needs to be tweaked in the follow-up PR which reworks the reporting script. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
clang-tidy made some suggestions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
f9e27e1
to
cd3d1ad
Compare
* Unify/standardize a few tags * Update ruby perf impact script to use names instead of column indices
cd3d1ad
to
efd0fb2
Compare
This PR is in preparation of a follow-up which will use tags in the evaluation for categorization.
This needs to be done in 2 steps due to the involvement of the repository state in the CI benchmark worklow.
Preparatory changes:
The last point might seem a bit unrelated. I added it because otherwise, our "full system" benchmark category would include only the many-task benchmark, which is quite far from Celerity's common use case currently.
We always wanted to extend this set anyway, so now I made a first step with a very simple iterative stencil benchmark that should better represent some common usage scenarios in terms of runtime system performance impact.
The goal is to be able to finish https://github.com/celerity/meta/issues/47 in the follow-up.