Skip to content

Commit

Permalink
Check that the ruby configuration isn't changed during the benchmark
Browse files Browse the repository at this point in the history
This produces an error like
.../yjit-bench/harness/harness.rb:70:in 'Object#run_benchmark': Benchmark altered ruby configuration! (RuntimeError)
 from ["ruby 3.4.0dev (2024-09-09T14:55:18Z master 1205f17125) [arm64-darwin23]", false]
   to ["ruby 3.4.0dev (2024-09-09T14:55:18Z master 1205f17125) +YJIT [arm64-darwin23]", true]
  • Loading branch information
rwstauner committed Sep 12, 2024
1 parent 06cf574 commit e55511c
Showing 1 changed file with 17 additions and 0 deletions.
17 changes: 17 additions & 0 deletions harness/harness.rb
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,19 @@

system('mkdir', '-p', File.dirname(OUT_CSV_PATH))

def ruby_configuration
[
# RUBY_DESCRIPTION can change based on enabled features
# (and the yjit-metrics reporting will abort if this changes).
# Interpolate to separate this string from the original object.
"#{RUBY_DESCRIPTION}",
# Explicitly check for YJIT toggling as we specifically don't want that.
defined?(RubyVM::YJIT) ? RubyVM::YJIT.enabled? : nil,
]
end

$original_ruby_configuration = ruby_configuration

puts RUBY_DESCRIPTION

def realtime
Expand Down Expand Up @@ -52,4 +65,8 @@ def run_benchmark(_num_itrs_hint, &block)
non_warmups_ms = ((non_warmups.sum / non_warmups.size) * 1000.0).to_i
puts "Average of last #{non_warmups.size}, non-warmup iters: #{non_warmups_ms}ms"
end

if ruby_configuration != $original_ruby_configuration
raise "Benchmark altered ruby configuration!\n from #{$original_ruby_configuration.inspect}\n to #{ruby_configuration.inspect}"
end
end

0 comments on commit e55511c

Please sign in to comment.