Skip to content

Commit

Permalink
Fix the issue where the first measurement in a microbenchmark is
Browse files Browse the repository at this point in the history
slightly slower, which is especially noticeable with quick benchmarks.

See the measurements (column A) from the
TrivialKotlinBenchmark#nothing microbenchmark from 100 runs before and
after this change in
https://docs.google.com/spreadsheets/d/1W6oxK_rM0KGzZBLxpax05766-hYyV2EYLw49dIs2uPo/edit?usp=sharing&resourcekey=0-3rhUgg9eSBekHXF8mOznng.

At head, every first measurement of a run (rows 1, 51, 101, 151, etc)
are in the range of ~5.5-6ns, while the median is 3.6ns. With this
change (the benchmark-first-measurement tab), this issue disappears.

The standard deviation at head is 0.42, while with this change it
reduces to 0.19.

Test: Existing tests.
Change-Id: Id6f1dac88d19665e9fa80d8e60144f38089f4cc2
  • Loading branch information
beloglazov committed Oct 29, 2024
1 parent afe2b3d commit 3083ea4
Showing 1 changed file with 5 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -319,6 +319,11 @@ class BenchmarkState internal constructor(phaseConfig: MicrobenchmarkPhase.Confi
profilerResult = phaseProfilerResult
}

// Warm up the metrics data structure to reduce the impact on the first measurement.
currentMetrics.captureStart()
currentMetrics.captureStop()
currentMetrics.captureInit()

currentMetrics.captureStart()
return true
}
Expand Down

0 comments on commit 3083ea4

Please sign in to comment.