[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU

From: Anthony Liguori
Subject: Re: [Qemu-devel] [ANNOUNCE] qemu-test: a set of tests scripts for QEMU
Date: Thu, 29 Dec 2011 10:26:17 -0600
User-agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv: Gecko/20110922 Lightning/1.0b2 Thunderbird/3.1.15

On 12/28/2011 11:21 AM, Avi Kivity wrote:
On 12/28/2011 06:42 PM, Anthony Liguori wrote:
In fact using linux as a guest negates that.  First of all, which linux
version? if it's fixed, you'll eventually miss functionality and need to
migrate.  If it keeps changing, so does your test, and it will keep

The kernel is a git submodule so it's a very specific version.  Yes,
we may need to bump the version down the road and obviously, if we
have to change any tests in the process, we can.

Having a full linux source as part of the build process detracts
somewhat from the advantages here.

Except that binaries can be made available. The initramfs is currently 512k while the kernel is about 5MB.

OTOH, a Fedora install runs in the multiple GB. Having kernels/initramfs for the dozen target types that QEMU supports is reasonable. Having Fedora installs for all of them probably isn't for most people.

But there's certain things that I still consider to be unit testing
(like basic networking tests) that I don't want to have to write with
qtest.  I'm not up for writing a TCP/IP stack in Python...

A ping test is not a unit test.

The ping test is already covered by kvm-autotest; just set up a config
that runs just that; after the initial run you'll have a guest installed
so it'll be quick.  If we have a DSL guest it'll even be very quick.

Achieving this with kvm-autotest is not easy. I doubt that DSL has an automated install mechanism and the step file based guest installation is fincky at best.

When we had the kvm-autotest day, none of the folks that tried to set it up who had never set it up before had something working by the end of the day. And that was with lmr available on IRC to answer questions.

There is a huge setup cost with kvm-autotest.

To test the various NIC emulations, you don't need a full TCP stack,
just like you didn't need to write an NTP implementation for qtest's rtc
test.  Instead you just poke values until it sends out a packet.  If you
want to test virtio-net with both direct and indirect buffers, you can
only do that with a unit test, you can't do it using a full linux guest
since it has its own ideas of when to use indirects and when to avoid
them (for example, it may choose to always avoid them).

I am not advocating that we don't write proper unit tests...

Acceptance testing is, "does Windows boot", "can I create three
virtio-serial devices".

Obviously, part of acceptance testing is, "does this set of functional
tests pass".

Seems like a very blurry line.  Especially as the functional test is
weaker than either qtest and kvm-autotest.  I now have to agree with the
others that it's duplicate functionality.  Does it really matter whether
you're creating an image by compiling Linux and assembling and
initramfs, or by downloading Fedora.iso and installing it?

Yes. One of them is built entirely from source and the other is doing something meant to be done on bare metal.

How do you test ARM guests with kvm-autotest? There is no such thing as an ISO installer for ARM. Are we going to download random images from the internet?

Even if there is an ISO, installing a guest with TCG will take many hours. My local qemu-test repo can now fully bootstrap an initramfs for non-x86 targets (which involves building gcc and uClibc). Start to finish it takes about 30 minutes regardless of the target (since the build runs natively).

Would you add live migration testing to qemu-test?  If yes, you're
duplicating some more.  If not, you're not doing functional or coverage
tests for that functionality.

I would add live migration schema testing, absolutely. I wouldn't add an acceptance test though.

An acceptance test with live migration would be creating a guest, running it, and then seeing it live migration worked.

I don't want to write a TCP/IP stack.  We aren't just grabbing a
random distro kernel.  We're building one from scratch configured in a
specific way.

How does that help?

Not sure I understand the question.


Anthony Liguori

In theory, we could even host the kernel source on git.qemu.org and
fork it to add more interesting things in the kernel (although I'd
prefer not to do this, obviously).

That way lies madness, though it could be argued that it lies all around us.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]