Progress reporting

[davidsantiago]

Because Criterium spends the greater of 1 minute and 60 evaluations of the test function, it would be nice if Criterium would periodically print something, perhaps a row of dots, so that you can see it is still working, and get an idea of the progress it is making. Could be controlled with a flag, I suppose.

[kumarshantanu]

+1 on this

In so many cases Criterium keeps on running and I have no clue when will it stop. A percentage indicator would be better.

Edit: In one case where I was trying to eval code from a Leiningen plugin the call to Criterium was simply not returning at all. So, barring that case Criterium does return in some time.

[jcf]

I was reading through the Criterium source and came across with-progress-reporting. It doesn't give you dots or a percentage, but you can see if progress is being made.

https://github.com/hugoduncan/criterium/blob/develop/src/criterium/core.clj#L752

It looks something like this:

(with-progress-reporting (./quick-bench (+ 1 2)))
Estimating sampling overhead
Warming up for JIT optimisations 10000000000 ...
  compilation occured before 287532 iterations
  compilation occured before 2299437 iterations
  compilation occured before 20119167 iterations
  compilation occured before 37364067 iterations
  compilation occured before 37651482 iterations
  compilation occured before 73003527 iterations
  compilation occured before 73290942 iterations
  compilation occured before 78464412 iterations
  compilation occured before 110367477 iterations
  compilation occured before 366741657 iterations
  compilation occured before 775445787 iterations
  compilation occured before 776020617 iterations
  compilation occured before 1329294492 iterations
  compilation occured before 1544280912 iterations
  compilation occured before 1546867647 iterations
  compilation occured before 1549454382 iterations
  compilation occured before 1559513907 iterations
  compilation occured before 1583369352 iterations
Estimating execution count ...
Sampling ...
Final GC...
Checking GC...
WARNING: Final GC required 7.469328032457214 % of runtime
Finding outliers ...
Bootstrapping ...
Checking outlier significance
Warming up for JIT optimisations 5000000000 ...
  compilation occured before 381349 iterations
  compilation occured before 762617 iterations
  compilation occured before 1143885 iterations
  compilation occured before 20207285 iterations
  compilation occured before 20969821 iterations
  compilation occured before 21351089 iterations
  compilation occured before 22494893 iterations
  compilation occured before 46514777 iterations
  compilation occured before 46896045 iterations
Estimating execution count ...
Sampling ...
Final GC...
Checking GC...
WARNING: Final GC required 173.525117607556 % of runtime
Finding outliers ...
Bootstrapping ...
Checking outlier significance
Evaluation count : 205891836 in 6 samples of 34315306 calls.
             Execution time mean : 1.099212 ns
    Execution time std-deviation : 0.141945 ns
   Execution time lower quantile : 0.938897 ns ( 2.5%)
   Execution time upper quantile : 1.255294 ns (97.5%)
                   Overhead used : 1.895269 ns

HTH.