Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit YJIT stats to the measured benchmark code #352

Merged
merged 1 commit into from
Dec 9, 2024
Merged

Conversation

rwstauner
Copy link
Contributor

@rwstauner rwstauner commented Dec 9, 2024

Reset stats at the beginning of the benchmark
and collect stats before loading any additional code.

You can see a difference with the invalidation count:

before:

$  MIN_BENCH_ITRS=1 WARMUP_ITRS=1 MIN_BENCH_TIME=0 RBENV_VERSION=ruby-master YJIT_BENCH_STATS=invalidation_count ruby --yjit-stats=quiet benchmarks/sequel/benchmark.rb
ruby 3.4.0dev (2024-12-04T21:34:16Z master 1c4dbb133e) +YJIT stats +PRISM [arm64-darwin23]
Command: bundle check 2> /dev/null || bundle install
The Gemfile's dependencies are satisfied
itr:   time invalidation_count
 #1:   31ms                  0
 #2:   30ms                  0
invalidation_count:         68

after:

$  MIN_BENCH_ITRS=1 WARMUP_ITRS=1 MIN_BENCH_TIME=0 RBENV_VERSION=ruby-master YJIT_BENCH_STATS=invalidation_count ruby --yjit-stats=quiet benchmarks/sequel/benchmark.rb
ruby 3.4.0dev (2024-12-04T21:34:16Z master 1c4dbb133e) +YJIT stats +PRISM [arm64-darwin23]
Command: bundle check 2> /dev/null || bundle install
The Gemfile's dependencies are satisfied
itr:   time invalidation_count
 #1:   31ms                  0
 #2:   30ms                  0
invalidation_count:          0

Reset stats at the beginning of the benchmark
and collect stats before loading any additional code.
@maximecb
Copy link
Contributor

maximecb commented Dec 9, 2024

@rwstauner for some reason the test is failing right at the end:

Building native extensions. This could take a while...
/opt/hostedtoolcache/Ruby/3.3.6/x64/lib/ruby/3.3.0/rubygems/ext/builder.rb:125:in `run': ERROR: Failed to build gem native extension. (Gem::Ext::BuildError)

Not immediately clear which gem is failing to build and why this happens so late in the process 🤔

EDIT: seeing that it happens with the --graph option. Guess it's an image library that's failing? If so, typical...

@maximecb maximecb merged commit 6d9c1d8 into main Dec 9, 2024
3 of 4 checks passed
@maximecb maximecb deleted the rwstauner/stats branch December 9, 2024 21:15
@rwstauner
Copy link
Contributor Author

The test started failing because the GitHub runner upgraded:
#353

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants