[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-devel] Improving QMP test coverage

From: Daniel P. Berrange
Subject: Re: [Qemu-devel] Improving QMP test coverage
Date: Thu, 27 Jul 2017 11:09:58 +0100
User-agent: Mutt/1.8.3 (2017-05-23)

On Thu, Jul 27, 2017 at 05:19:57PM +0800, Fam Zheng wrote:
> On Thu, 07/27 10:14, Markus Armbruster wrote:
> > This brings some advantages of "verify output with diff" to tests that
> > verify with code.  Improvement if it simplifies the verification code.
> > 
> > I'd still prefer *no* verification code (by delegating the job to diff)
> > for tests where I can get away wit it.
> Python based iotests can be (re)done in such a way that they print actual logs
> (interactions with qtest/monitor, stdout/stderr of QEMU, etc) instead of the
> current dot dot dot summary, then we automatically have diff based 
> verification,
> no?

The python test 149 that I wrote does exactly that. There's no reason why
the others couldn't do the same.

> One thing I feel painful with bash iotests is how harder it is to write
> complicated test scenarios such as migration, incremental backup, etc.

Yes, shell is an awful language if you need non-trivial control
logic or data structures

> On the other hand the iotests are more difficult to debug when things go wrong
> because it eats the output which, if done with shell, should be very easy to
> get.

Even if the python tests are not doing verify-by-diff, it should be fairly
easy to wire up an env variable IOTESTS_DEBUG=1 which would force them
to propagate stdout/err of all commands run (or at least save it to a
log file somewhere).

|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

reply via email to

[Prev in Thread] Current Thread [Next in Thread]