Re: -warmup

[Werner LEMBERG]

>> Should I proceed to detect outliers?  Since we do not get the same
>> error rate consistently, I think we will not find the target we
>> expected by outliers.
> 
> Why do you think so?  Please explain your reasoning.  Just remember
> that backup processes (like cleaning up the hard disk, running some
> cron jobs, etc.) can pop up anytime, thus influencing the result.
> Such spontaneous events have to be eliminated.
> 
> Have you actually tried something along the method I suggested?

BTW, here is another repository that provides a framework for
benchmarks:

  https://github.com/sharkdp/hyperfine

It looks quite nice, AFAICS – maybe you can check its code for
identifying outliers and the like.


    Werner