Re: [PATCH v2 7/8] simplebench/bench-backup: add --count and --no-initial-run

[John Snow]

On 3/4/21 5:17 AM, Vladimir Sementsov-Ogievskiy wrote:
> Add arguments to set number of test runs per table cell and to disable
> initial run that is not counted in results.
>
> It's convenient to set --count 1 --no-initial-run to fast run test
> onece, and to set --count to some large enough number for good
> precision of the results.
>
> Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> ---
>   scripts/simplebench/bench-backup.py | 10 +++++++++-
>   1 file changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/scripts/simplebench/bench-backup.py 
> b/scripts/simplebench/bench-backup.py
> index a2120fcbf0..519a985a7f 100755
> --- a/scripts/simplebench/bench-backup.py
> +++ b/scripts/simplebench/bench-backup.py
> @@ -155,7 +155,9 @@ def bench(args):
>                   'qemu-binary': path
>               })
>   
> -    result = simplebench.bench(bench_func, test_envs, test_cases, count=3)
> +    result = simplebench.bench(bench_func, test_envs, test_cases,
> +                               count=args.count,
> +                               initial_run = not args.no_initial_run)

The double negative feels odd; "initial_run = args.initial_run" would 
read better and avoid changing behavior, but maybe that's intentional.

>       with open('results.json', 'w') as f:
>           json.dump(result, f, indent=4)
>       print(results_to_text(result))
> @@ -211,4 +213,10 @@ def __call__(self, parser, namespace, values, 
> option_string=None):
>      both: generate two test cases for each src:dst pair''',
>                      default='direct', choices=('direct', 'cached', 'both'))
>   
> +    p.add_argument('--count', type=int, default=3, help='''\
> +Number of test runs per table cell''')
> +
> +    p.add_argument('--no-initial-run', action='store_true', help='''\
> +Don't do initial run of test for each cell which doesn't count''')
> +
>       bench(p.parse_args())