Re: [PATCH] meson: adjust timeouts for some slower tests

[Daniel P . Berrangé]

On Tue, Feb 09, 2021 at 06:45:41PM +0100, Paolo Bonzini wrote:
> Adjust the timeouts for the longest running tests.  These are the
> times that I measured and the corresponding timeouts.  For generic
> qtests, the target that reported the longest runtime is included.
> 
> unit tests:
>     test-crypto-tlscredsx509        13.15s   60s
>     test-crypto-tlssession          14.12s   60s

The default meson timeout is 30 seconds which is enough for these
tests. Of course larger timeouts give more headroom.

If running on particularly slow environemnt (eg under TCG)
then it would be expecte dto instead use "meson --timeout-multiplier=10"
to make everything have more headroom.

So I wonder what threshold we need the extra headroom above
30 seconds in the default config ?


> qtests:
>     ide-test                        13.65s   60s
>     npcm7xx_watchdog_timer-test     15.02s   60s
>     qos-test                        21.26s   60s   (i386)
>     ahci-test                       22.18s   60s
>     pxe-test                        26.51s   90s
>     boot-serial-test                28.02s   90s   (sparc)
>     prom-env-test                   28.86s   90s
>     bios-tables-test                50.17s   90s   (aarch64)
>     test-hmp                        57.15s   90s   (aarch64)
>     npcm7xx_pwm-test                71.27s   120s
>     migration-test                  97.09s   150s  (aarch64)
>     qom-test                        139.20s  180s  (aarch64)
> 
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  tests/meson.build       |  6 ++++++
>  tests/qtest/meson.build | 15 +++++++++++++++
>  2 files changed, 21 insertions(+)
> 
> diff --git a/tests/meson.build b/tests/meson.build
> index 0c939f89f7..fde21b6c8f 100644
> --- a/tests/meson.build
> +++ b/tests/meson.build
> @@ -237,6 +237,11 @@ test_env = environment()
>  test_env.set('G_TEST_SRCDIR', meson.current_source_dir())
>  test_env.set('G_TEST_BUILDDIR', meson.current_build_dir())
>  
> +slow_tests = {
> +  'test-crypto-tlscredsx509': 2,
> +  'test-crypto-tlssession': 2
> +}
> +
>  foreach test_name, extra: tests
>    src = [test_name + '.c']
>    deps = [qemuutil]
> @@ -254,6 +259,7 @@ foreach test_name, extra: tests
>         env: test_env,
>         args: ['--tap', '-k'],
>         protocol: 'tap',
> +       timeout: 30 * slow_tests.get(test_name, 1),
>         suite: ['unit'])
>  endforeach
>  
> diff --git a/tests/qtest/meson.build b/tests/qtest/meson.build
> index 7e082be6fb..b1f8790b75 100644
> --- a/tests/qtest/meson.build
> +++ b/tests/qtest/meson.build
> @@ -4,6 +4,20 @@ if not config_host.has_key('CONFIG_POSIX')
>    subdir_done()
>  endif
>  
> +slow_qtests = {
> +  'ahci-test' : 2,
> +  'bios-tables-test' : 3,
> +  'boot-serial-test' : 3,
> +  'ide-test' : 2,
> +  'migration-test' : 5,
> +  'npcm7xx_pwm-test': 4,
> +  'npcm7xx_watchdog_timer-test': 2,
> +  'prom-env-test' : 3,
> +  'pxe-test' : 3,
> +  'qos-test : 2,
> +  'qom-test' : 6,
> +  'test-hmp' : 3,
> +]
>  qtests_generic = [
>    'cdrom-test',
>    'device-introspect-test',
> @@ -272,6 +286,7 @@ foreach dir : target_dirs
>           env: qtest_env,
>           args: ['--tap', '-k'],
>           protocol: 'tap',
> +         timeout: 30 * slow_qtests.get(test_name, 1),
>           suite: ['qtest', 'qtest-' + target_base])
>    endforeach
>  endforeach
> -- 
> 2.29.2
> 
> 

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|