Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add logging message to notify user of skipped benchmarks #316

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

bclodius
Copy link

@bclodius bclodius commented Oct 9, 2023

Background

Hello!

I was recently running a number of benchmarks to analyze a migration to different python3 versions as well as different server infrastructure.

In my first 30-60 minutes playing with this library I was very confused that my results would only show benchmarks for comprehensions and sqlglot. After some breadcrumbs and print statements I finally came to realize that a large majority of the benchmarks require >=3.8. I was testing on a version of python 3.7.

I was hoping this small enhancement would help other users in future to prevent hair pulling and headaches!

Testing

Before

...
pyperformance run -b all
...
( 1/2) creating venv for benchmark (comprehensions)
...
( 2/2) creating venv for benchmark (sqlglot)
...

After

pyperformance run -b all
WARNING:root:Benchmark: 2to3 is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_generators is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_cpu_io_mixed is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_io is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_memoization is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_cpu_io_mixed is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_io is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_memoization is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_cpu_io_mixed_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_io_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_memoization_tg is skipped because python version required is >=3.11 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_cpu_io_mixed_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_io_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: async_tree_eager_memoization_tg is skipped because python version required is >=3.12 but found 3.7.16
WARNING:root:Benchmark: asyncio_tcp is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: asyncio_tcp_ssl is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: asyncio_websockets is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: concurrent_imap is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: coroutines is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: coverage is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: gc_traversal is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: gc_collect is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: generators is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: chameleon is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: chaos is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: crypto_pyaes is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: dask is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: deepcopy is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: deltablue is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: django_template is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: dulwich_log is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: docutils is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: fannkuch is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: float is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: genshi is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: go is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: hexiom is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: html5lib is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: json_dumps is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: json_loads is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: logging is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: mako is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: mdp is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: meteor_contest is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: nbody is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: nqueens is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pathlib is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_dict is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_list is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pickle_pure_python is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pidigits is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pprint is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: pyflate is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: python_startup is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: python_startup_no_site is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: raytrace is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_compile is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_dna is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_effbot is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: regex_v8 is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: richards is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: richards_super is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: scimark is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: spectral_norm is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlalchemy_declarative is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlalchemy_imperative is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sqlite_synth is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: sympy is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: telco is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: tomli_loads is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: tornado_http is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: typing_runtime_protocols is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpack_sequence is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle_list is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: unpickle_pure_python is skipped because python version required is >=3.8 but found 3.7.16
WARNING:root:Benchmark: xml_etree is skipped because python version required is >=3.8 but found 3.7.16

pyperformance/cli.py Outdated Show resolved Hide resolved
pyperformance/cli.py Outdated Show resolved Hide resolved
@bclodius bclodius requested a review from hugovk October 9, 2023 20:53
@bclodius
Copy link
Author

bclodius commented Oct 9, 2023

@hugovk thanks for your feedback, I believe I've incorporated it all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants