Re: [Qemu-devel] [RFC] Future goals for autotest and virtualization tests

[Ademar Reis]

On Thu, Mar 08, 2012 at 04:46:09PM +0100, Kevin Wolf wrote:
> Am 08.03.2012 15:56, schrieb Anthony Liguori:
> >> I particularly agreed with basically everything you said on that
> >> discussion regarding test simplification (I had just joined the
> >> team back then). To me, autotest has been focusing on QE-level,
> >> leaving the developer-level test requirements out. Now we're
> >> attacking this new front, and a lot of the requirements are
> >> indeed from that discussion.
> > 
> > If you want to talk about this in terms of "requirements", my requirement 
> > is for 
> > "developer-level" tests to live in qemu.git and be integrated into make 
> > check.
> 
> Actually, I think make check should be something that runs _really_
> quickly. We'll probably want another make check-full or something that
> runs more extensive tests.

I like make check because it's part of make, but I would suggest:

$ test-runner <dir> | <file> ...

This would allow use-cases such as:

[qemu]$ test-runner tests.d/block/
  (run all block tests)

[qemu]$ test-runner tests.d/core/whatever-i-want-to-test.sh
  (run one individual test)

[qemu]$ test-runner tests.d/
  (run all tests)

[qemu]$ test-runner tests.d/devices/*pci*
  (you get the idea)

And, of course:
[qemu]$ test-runner --remote=autotest.qemu.org tests.d/block

(a test path/name is its ID to both the local test runner and the
remote server)

Cheers,
  - Ademar

> 
> That's the reason why the patch I proposed earlier today runs only a few
> qemu-iotests cases during make check. The whole thing would already take
> too long for everyone to use it. If people run some quick sanity tests
> for all of qemu plus the full tests in the subsystem in which they're
> making changes, I think that would be the best we can realistically expect.
> 
> Kevin

-- 
Ademar de Souza Reis Jr.
Red Hat

^[:wq!