Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🛤️ Tracking: Periodic Perf Benchmarking #787

Open
3 tasks
j-lanson opened this issue Jan 14, 2025 · 0 comments
Open
3 tasks

🛤️ Tracking: Periodic Perf Benchmarking #787

j-lanson opened this issue Jan 14, 2025 · 0 comments
Labels
product: hc Relates to the core "hc" binary product: project Relates to the project itself product: website Relates to the Hipcheck website tracking-issue This issue tracks a larger task. type: enhancement New feature or request

Comments

@j-lanson
Copy link
Collaborator

j-lanson commented Jan 14, 2025

Similar to how Rust does it, we should have a page on our website that records and displays changes to the performance of Hipcheck over time, as the repository changes.

We should have an xtask called benchmark that takes a config file of a list of targets and executes hc check against them with benchmarking turned on. Usage: cargo xtask benchmark <CONFIG_FILE> [--out <OUT_FILE>], where --out is optional. This will write a file to site/static/perf/ with the name <TIMESTAMP>.csv. The execution should record our internal benchmarking stats, as well as linux ram statistics like "max resident set size". Use this Rust benchmarking library as well.

The config file should be a .kdl file, and target some basic repos as well as some of hipchecks own dependencies.

  • - Implement benchmark xtask

Once we have this, we need a CI cron job action that will invoke the benchmark xtask once a week and create a PR requesting to merge the new file in site/static/perf into main.

  • - Add CI job to execute benchmark xtask

Finally, we need to add a <SITE>/perf page to display the data in these .csv files.

  • - Add /perf page to site to display benchmark xtask output
@j-lanson j-lanson added type: enhancement New feature or request tracking-issue This issue tracks a larger task. product: hc Relates to the core "hc" binary product: website Relates to the Hipcheck website product: project Relates to the project itself labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
product: hc Relates to the core "hc" binary product: project Relates to the project itself product: website Relates to the Hipcheck website tracking-issue This issue tracks a larger task. type: enhancement New feature or request
Projects
Status: Todo
Status: No status
Development

No branches or pull requests

1 participant