Re: [Qemu-devel] [PATCH 2/3] qtest: enable qtest for most targets

[Blue Swirl]

On Tue, Apr 17, 2012 at 20:59, Peter Maydell <address@hidden> wrote:
> On 17 April 2012 21:43, Blue Swirl <address@hidden> wrote:
>> On Tue, Apr 17, 2012 at 20:31, Peter Maydell <address@hidden> wrote:
>>> Well, it could. But we should make that decision based on whether it
>>> makes sense and has a use case for actual users of the board, not
>>> because we're trying to get away with not having a setup that lets
>>> us properly unit test devices.
>>
>> I disagree. I see many benefits in qtest and very little downsides in
>> changes like these.
>>
>> qtest is a tool to let developers test the changes they make to
>> devices, so breakages shouldn't be so common. This should improve the
>> development process in QEMU tremendously.
>
> I'm entirely in favour of having a decent testing framework so we
> can easily write unit tests for device models. What I don't understand
> is why a developer only unit testing tool seems to require changes
> to user visible behaviour across dozens of board models. Something
> is wrong in its design somewhere, and I think that's what I'm
> objecting to as much as to the specific detail of what's being
> changed in this patch. (And I don't want us to add lots of tests
> and/or changes to the code before we fix whatever the problem is.)

Even if we had this single device test setup, board level testing
would be useful. It shouldn't test the devices anymore in this case
but how the board works.

 I think ideally the test sequence should go like this:
1. QEMU internals (block, qint etc)
2. Device level (only once for each device, list of devices collected
from all architectures and boards)
3. Board level (all architectures, all boards, just high level stuff
like test that the devices are in place, IRQ routing etc.)

But since a lot of devices still have target dependencies (at least
bus width), testing the devices when they are in a board like now is
not a bad choice either except for the performance hit.

>
> -- PMM