Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a benchmark for the Azure CLI. #90

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

ericsnowcurrently
Copy link
Member

This is part of CPython performance work that @gvanrossum and I are starting. We want to use pyperformance to run our benchmarks (and profiling, but that's a separate issue). As there isn't a mechanism to plug in custom benchmarks, I'm putting up this PR.

I suppose there's the question of whether this is an appropriate benchmark for the suite. I think so. It represents a workload, based on concrete usage patterns for a widely used CLI app, that does not seem to be covered by other benchmarks. Note that at the moment I'm cheating a little by running tests out of the azure-cli test suite. However, I expect the tests exercise the code in a very similar way to common usage of the actual application. If there are concerns about inclusion I'm fine with leaving it out.

We will be using this benchmark (and a few others we will probably add) extensively.

(Either way, it would still be nice to have a way to plug in custom benchmarks externally. See #89.)

@ericsnowcurrently
Copy link
Member Author

Hmm...

Execute: 'C:\Users\RUNNER~1\AppData\Local\Temp\tmpg12c4haj\Scripts\python.exe' -m azdev setup --cli 'D:\a\pyperformance\pyperformance\pyperformance\benchmarks\data\azure-cli'
Fatal Python error: _Py_HashRandomization_Init: failed to get random numbers to initialize Python
Python runtime state: preinitialized

@ericsnowcurrently
Copy link
Member Author

FYI, using azure-cli on 3.9+ is currently blocked: Azure/azure-cli#16611.

Base automatically changed from master to main March 24, 2021 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant