Re: "make check-acceptance" takes way too long

[Daniel P . Berrangé]

On Mon, Aug 02, 2021 at 01:47:37PM +0100, Alex Bennée wrote:
> 
> Daniel P. Berrangé <berrange@redhat.com> writes:
> 
> > On Fri, Jul 30, 2021 at 04:12:27PM +0100, Peter Maydell wrote:
> >> "make check-acceptance" takes way way too long. I just did a run
> >> on an arm-and-aarch64-targets-only debug build and it took over
> >> half an hour, and this despite it skipping or cancelling 26 out
> >> of 58 tests!
> >> 
> >> I think that ~10 minutes runtime is reasonable. 30 is not;
> >> ideally no individual test would take more than a minute or so.
> >> 
> >> Output saying where the time went. The first two tests take
> >> more than 10 minutes *each*. I think a good start would be to find
> >> a way of testing what they're testing that is less heavyweight.
> >
> > While there is certainly value in testing with a real world "full" guest
> > OS, I think it is overkill as the default setup. I reckon we would get
> > 80-90% of the value, by making our own test image repo, containing minimal
> > kernel builds for each machine/target combo we need, together with a tiny
> > initrd containing busybox. This could easily be made to boot in 1 second,
> > which would make 'make check-acceptance' waaaaay faster, considering how
> > many times we boot a guest. This would also solve our problem that we're
> > pointing to URLs to download these giant images, and they're frequently
> > break URLs.
> 
> It's been discussed before but previously the worry has been the hassle
> of maintaining such images along with such tediousness as ensuring GPL
> compliance. We've outsourced that problem to the upstream.

I don't recall discussing that directly - only discussions around
hosting images / files from other distros on our own infra, that
does indeed create a compliance burden.

This is why I suggested /strictly/ nothing more than kernel+busybox
built from source ourselves, probably using debian cross compilers.

The busybox stuff would only need to be built once per architecture.
The kernel would potentially need more builds to cope with machine
board specific configs. We would not need to continually track new
releases - we can fix on specific kernel + busybox versions for as
long as they cope with the targets/archs we need.

I'd expect it all to be done in a gitlab repo, with a CI job to
publish the results, never any manual builds, so that we ensure
license compliance.

Of course the main problem is someone doing the leg work to get
such a system up & running initially to prove the concept.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|