[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

From: Lucas Meneghel Rodrigues
Subject: Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU
Date: Thu, 29 Dec 2011 20:11:46 -0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111222 Thunderbird/9.0

On 12/29/2011 05:04 PM, Peter Maydell wrote:
On 29 December 2011 18:35, Anthony Liguori<address@hidden>  wrote:
On 12/29/2011 11:49 AM, Peter Maydell wrote:
The next obvious question is: are we going to make a serious attempt?
(For instance, in a hypothetical tests-required world, would we
tell those nice folks from Samsung "no you can't land your
Exynos patches unless you write 9000+ lines of test cases" ?

The virtio-serial test case I posted was 50 lines in qemu-test.  The
virtio-serial driver is about ~1500 LOC.  That's about 3%.

This just means it's testing only a fraction of what virtio-serial
actually implements (as you note yourself in the comments for the
test case). For serious coverage you'd also need to cover reading(!),
larger quantities of data than single lines, what happens when one end
feeds in data but the other end isn't reading, vmstate save/load,
multiple simultaneous ports, behaviour on close-and-reopen,
whether it works on bigendian targets, benchmarking to identify
possible performance regressions, etc etc. I think that by the time
you've done all that you'll be closer to 1500 lines than 50.

Agreed. There's a reason why virtio-serial testing in kvm autotest is 2.1 k LOC for the test itself and 0.9 k LOC for the auxiliary guest script. Most of the scenarios you've described are covered on kvm autotest version.

I would expect that we at least have some sort of test that could verify
that the Exynos platform more or less worked as expected.  If that was just
booting a Linux kernel, that would be fine by me.

Yes. There's a large range between "no tests required at all" (essentially
what we have now) and "an equivalent level of device testing to what you
would carry out before sending your hardware design out to be fabbed into
silicon" (which I hope we'd all agree would be ludicrously high for QEMU);
I'm trying to establish where we're attempting to set our bar.

I agree that we'd get a lot of bang-for-the-buck out of basic automated
"boot the guest and prod it" testing that covered most of our platforms.

And that sort of automated testing can be done right now with KVM autotest. Granted, I never had time to make it work with ARM, but if people are interested, we are happily accepting patches :)

reply via email to

[Prev in Thread] Current Thread [Next in Thread]