[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: R
From: |
Eric Blake |
Subject: |
Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run |
Date: |
Tue, 7 May 2019 10:50:22 -0500 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 |
On 5/7/19 10:22 AM, Thomas Huth wrote:
> On 07/05/2019 15.22, Markus Armbruster wrote:
>> Thomas Huth <address@hidden> writes:
>>
>>> Currently, all tests are in the "auto" group. This is a little bit
>>> pointless.
>>> OTOH, we need a group for the tests that we can automatically run during
>>> "make check" each time, too. Tests in this new group are supposed to run
>>> with every possible QEMU configuration, for example they must run with every
>>> QEMU binary (also non-x86), without failing when an optional features is
>>> missing (but reporting "skip" is ok), and be able to run on all kind of host
>>> filesystems and users (i.e. also as "nobody" or "root").
>>> So let's use the "auto" group for this class of tests now. The initial
>>> list has been determined by running the iotests with non-x86 QEMU targets
>>> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including
>>> macOS and FreeBSD).
>>
>> I wonder whether we should additionally limit "make check" to "quick"
>> tests. How slow are the non-quick auto tests for you?
>
> I already sorted out some of the tests that run veeeery long, since the
> run time on gitlab, cirrus-ci and travis is limited. "make check-block"
> currently takes 3 minutes on my laptop, I think that's still ok?
>
> When I run the tests from the auto group that are not in the quick
> group, I currently get:
>
My personal threshold is about 5 seconds for quick, so:
> 003 1s ...
> 007 2s ...
Should these be moved to quick?
> 013 5s ...
this one is borderline
> 014 15s ...
> 015 9s ...
Definitely not quick, but if you think they are still okay for auto, I
can live with that.
> 022 1s ...
Another candidate for quick?
> 023 18s ...
Even longer than 14. Okay for auto?
etc.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3226
Virtualization: qemu.org | libvirt.org
signature.asc
Description: OpenPGP digital signature
- [Qemu-block] [PATCH v3 0/7] tests/qemu-iotests: Run basic iotests during "make check", Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 2/7] tests/qemu-iotests/check: Pick a default machine if necessary, Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 1/7] tests/qemu-iotests/005: Add a sanity check for large sparse file support, Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 3/7] tests/qemu-iotests: Do not hard-code the path to bash, Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 4/7] cirrus / travis: Add gnu-sed and bash for macOS and FreeBSD, Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 5/7] tests/qemu-iotests: Remove the "_supported_os Linux" line from many tests, Thomas Huth, 2019/05/02
- [Qemu-block] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Thomas Huth, 2019/05/02
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Markus Armbruster, 2019/05/07
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Thomas Huth, 2019/05/07
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run,
Eric Blake <=
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Thomas Huth, 2019/05/08
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Thomas Huth, 2019/05/10
- [Qemu-block] [PATCH v4] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Thomas Huth, 2019/05/10
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Markus Armbruster, 2019/05/10
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Kevin Wolf, 2019/05/10
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Markus Armbruster, 2019/05/10
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Kevin Wolf, 2019/05/10
- Re: [Qemu-block] [Qemu-devel] [PATCH v3 6/7] tests/qemu-iotests/group: Re-use the "auto" group for tests that can always run, Markus Armbruster, 2019/05/08
[Qemu-block] [PATCH v3 7/7] tests: Run the iotests during "make check" again, Thomas Huth, 2019/05/02