Re: [Qemu-devel] [PATCH 0/3] script for crash-testing -device

[Thomas Huth]

On 22.03.2017 20:13, Eduardo Habkost wrote:
> On Wed, Mar 22, 2017 at 01:00:49PM -0300, Eduardo Habkost wrote:
>> This series adds scripts/device-crashtest.py, that can be used to
>> crash-test -device with multiple machine/accel/device
>> combinations.
>>
>> The script found a few crashes on some machines/devices. A dump
>> of existing cases can be seen here:
>>   https://gist.github.com/ehabkost/503b0af0375f0d98d3e84017e8ca54eb
>>
>> The script contains a whitelist that can also be useful as
>> documentation of existing ways -device can fail or crash.
>>
>> Note that the script takes a few hours to run on the default mode
>> (testing all accel/machine/device combinations), but the "-r N"
>> option can be used to make it only test N random samples.

Wow, impressive script, that must have been a lot of work 'til you've
got it in a usable shape with that huge whitelist!

> Something I forgot to mention: I would like to run some subset of
> these tests on "make check", but I don't know how we could choose
> that subset. We could run, e.g., 100 random samples, but I am not
> sure we really want to make "make check" non-deterministic.

Maybe limit the tests to the devices that have a high chance to work on
different machines? ... that means primarily PCI, ISA and USB devices, I
guess.

 Thomas