Re: [Qemu-devel] Improving QMP test coverage

[Stefan Hajnoczi]

On Thu, Jul 13, 2017 at 05:28:52PM +0200, Markus Armbruster wrote:
> What can we do to improve QMP testing?  Sadly, I don't have the master
> plan ready.  I can tell people their new code needs to come with tests,
> but that won't help much unless subsystem maintainers pick up the habit,
> too.  There are a few obvious tests to write for old code, such as a
> generic test of query-like commands without arguments and without side
> effects, similar to what test-hmp.c does for HMP command info (I hope to
> get around to that one).  But for much of the old code, we don't even
> know where the test coverage holes are.
> 
> Ideas anyone?

It makes sense for maintainers to ask that new QMP commands come with
comprehensive tests.

For me the main question is how to begin?  What is the easiest and
preferred way to write QMP command test cases?

Today the most common approach is a qtest test case that sends commands
and verifies that the expected response is returned.  This approach
works but we could benefit from a discussion about the alternatives
(e.g.  qemu-iotests style shell scripts with output diffing).

If it's easy to write tests then developers will contribute more tests.

Stefan

signature.asc
Description: PGP signature