-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CI: Add Benchmarking #46
Conversation
1e81d67
to
6c2389c
Compare
a6fe9e4
to
ab16d37
Compare
ab16d37
to
fa34e01
Compare
Ok, it looks like the basic benchmarking is working. You can see that the commit with an intentional performance regression failed CI. Two things I'd like to add:
I'd like to just leave this with a test for finite difference as a template later expansion and improvement. I think the PR is ready to go now |
47c0f36
to
3b461ca
Compare
a9457f0
to
55f8c8e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is great, and will hopefully encourage contributions.
Hey Andy, I've added some logging to an experiment and found that some derivative methods can be slow. This isn't a surprise, but the enterprise solution is to add benchmarking to CI so that we can start quantifying improvements. This uses Airspeed Velocity (
asv
), the same benchmarking package as pandas.Like most PRs that deal with CI, this one may have a variety of commits to try to get it working. I don't know yet how to feed asv previous benchmark results so that it can know performance improvements/regressions.