[Qemu-devel] make check speed (was: Re: [PATCH for-2.10] boot-serial-test: prefer tcg accelerator)

[Thomas Huth]

On 23.08.2017 09:16, Cornelia Huck wrote:
> On Wed, 23 Aug 2017 10:29:07 +1000
> David Gibson <address@hidden> wrote:
> 
>> On Tue, Aug 22, 2017 at 01:48:15PM +0200, Cornelia Huck wrote:
>>> On Tue, 22 Aug 2017 21:20:46 +1000
>>> David Gibson <address@hidden> wrote:
>>>   
>>>> Obviously it's not a thing to fix right now, but I've really been
>>>> thinking that none of the tests should use this "TCG or KVM" stuff.
>>>> They should instead be run with *both* options - or at least the ones
>>>> that are available on the host.  
>>>
>>> Having one test as a 'smoke test' that is run for everything available
>>> sounds like a good idea, and the boot-serial test may be a good
>>> candidate for that.
>>>
>>> I would not want to run every test with every accelerator, however, as
>>> this makes 'make check' even slower than it is now. (Although it may be
>>> useful to be able to trigger 'run everything' tests on some dedicated
>>> test machines.)  
>>
>> I'd be fine with only running the full matrix on a "make check-harder"
>> or whatever, target.  But I'd like the option to be there.  Sometimes
>> (like when preparing a pull request) a slower check is an acceptable
>> cost for better coverage.
> 
> make check with one smoke test + make check-harder (like the name :)
> sounds like a good combination.

While we're at it: I'd like to have a "make check-fast", too. Sometimes
the normal "make check" is already too slow, e.g. while developing new
patches, I sometimes just want to do a very quick sanity test to see
whether I broke some basic things or not, and only do the "make check"
before I submit my patches.
So we would get three stages:

- make check-fast => For very, very quick sanity tests only

- make check => E.g. has to be run before submitting patches

- make check-harder => might run a very long time, so best suited for
                       nightly regression tests etc.?

Does that sound reasonable? And the crucial question: Who is going to
implement the basic framework for this?

 Thomas