Re: venv for python qtest bits? (was: Re: [PATCH 11/12] acpi/tests/bits: add README file for bits qtests)

[Michael S. Tsirkin]

On Tue, Jun 28, 2022 at 11:18:14AM +0100, Daniel P. Berrangé wrote:
> On Tue, Jun 28, 2022 at 06:07:13AM -0400, Michael S. Tsirkin wrote:
> > On Tue, Jun 28, 2022 at 11:04:30AM +0100, Daniel P. Berrangé wrote:
> > > If it is actually booting a real guest image (from biosbits) and 
> > > interacting
> > > with it, then it does feel like the scope of this testing is more 
> > > appropriate
> > > to QEMU's avocado framework than qtest, especially given the desire to use
> > > python for it all.
> > > 
> > > With regards,
> > > Daniel
> > 
> > I feel avocado is directed towards booting full fledged guest OS.
> 
> That's essentially what this is doing - its a custom guest OS rather
> than a common distro IIUC

yes but then so is bios tables test - we generate the disk on the fly.

> > It makes it much easier to figure out guest issues but it also
> > prone to false positives and is harder to debug as a result.
> > Booting a minimal image like this shouldn't require that.
> 
> Well avocado is as reliable as the tests you write for it. The problems
> are largely around the images being used in avocado.

The idea of downloading large images as part of the test as opposed to
part of setup is a large part of what makes it flaky - due to
running into unpredictable latency and errors on both the internet and
local IO side of things.

> If the biosbits
> testing system is reliable, then avocado will be too, and if they not
> reliable, then it will affect qtest too.

A larger system has more potential for bugs ;) I'm with Ani here. Using
a system that is overkill will just make debugging more painful than it
needs to be.

> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|