Re: [Qemu-devel] make check speed

[Thomas Huth]

On 23.08.2017 13:51, Lukáš Doktor wrote:
> Dne 23.8.2017 v 10:35 Thomas Huth napsal(a):
>> On 23.08.2017 10:01, Paolo Bonzini wrote:
>>> On 23/08/2017 09:49, Thomas Huth wrote:
>>>> While we're at it: I'd like to have a "make check-fast", too. Sometimes
>>>> the normal "make check" is already too slow, e.g. while developing new
>>>> patches, I sometimes just want to do a very quick sanity test to see
>>>> whether I broke some basic things or not, and only do the "make check"
>>>> before I submit my patches.
>>>> So we would get three stages:
>>>>
>>>> - make check-fast => For very, very quick sanity tests only
>>>>
>>>> - make check => E.g. has to be run before submitting patches
>>>>
>>>> - make check-harder => might run a very long time, so best suited for
>>>>                        nightly regression tests etc.?
>>>>
>>>> Does that sound reasonable? And the crucial question: Who is going to
>>>> implement the basic framework for this?
>>>
>>> There's already make check-unit or make check-qtest-x86_64 depending on
>>> what you're working on.
>>
>> True. And I just learned that you can also already set the SPEED
>> variable to either "quick" or "slow" and that we're already using
>> g_test_quick() and g_test_slow() in a couple of places to check this. So
>> the framework for running quick vs. thorough tests is already there ...
>> we just might want to add this to some more tests, I guess...
>>
>> Question for the maintainers and the test automation folks: Is anybody
>> already running "make check SPEED=slow" or is this just rather an
>> unheard-of way of running the tests?
>>
>>> If you have a many-core machine, of course, there's no simpler solution
>>> than throwing more CPUs at it. :)
>>
>> Is it safe nowadays to run "make check -j4" for example? Last time I
>> tried (maybe 1 or 2 years ago), there were still issues since some tests
>> were using hard-coded temporary file names, so the parallel tests were
>> disturbing each other, for example...
>>
> 
> Actually the `.travis.yml` defines `MAKEFLAGS="-j3"`, which results in `make 
> check` being executed with 3 threads...
> 
> I was actually looking at the increasing number of failed travis builds and 
> it seems to be related to the fluctuating performance. Running `nice -n 20 
> make check -j 12` with `nice -n 5 stress -c 20` in background results in the 
> same kind of failures:
> 
>     File '/tmp/qemu/include/qapi/qmp/qobject.h'
>     Lines executed:0.00% of 9
>     **
>     ERROR:tests/prom-env-test.c:42:check_guest_memory: assertion failed 
> (signature == MAGIC): (0x7c7f1b78 == 0xcafec0de)

I think you're simply running into timeout issues here since you're
likely overloading the host, I guess. Or does your host have that many CPUs?
If the timeouts are really an issue here, we simply might have to
increase the timeout values a bit again...

 Thomas

signature.asc
Description: OpenPGP digital signature