Re: [Qemu-devel] State of QEMU CI as we enter 4.0

[Alex Bennée]

Daniel P. Berrangé <address@hidden> writes:

> On Fri, Mar 15, 2019 at 09:34:27AM +0000, Alex Bennée wrote:
>>
>> Daniel P. Berrangé <address@hidden> writes:
>>
>> > On Thu, Mar 14, 2019 at 03:57:06PM +0000, Alex Bennée wrote:
>> >> Docker Images
>> >> =============
>> >>
>> >> The addition of docker has unlocked the ability to build a lot more
>> >> tests as well as compile testing on a much wider range of distros. I
>> >> think there are two outstanding areas that need improvement
>> >>
>> >> Daniel has been looking at building and hosting the images somewhere.
>> >> This would be useful as it would stop us slamming the distros
>> >> repositories constantly rebuilding the same images and also help reduce
>> >> the time to test.
>> >
>> > My intent was/still is to make use of quay.io for hosting prebuilt
>> > images.
>> >
>> > As well as avoiding repeated builds for developers it means that
>> > developers can be gauranteed to actually be testing with the same
>> > content that the automated CI did. Currently everyone using the
>> > docker images potentially has slightly different environment as
>> > it depends on what packages were in the repos when they built
>> > the image locally. This is very bad for reproducability.
>> >
>> > Libvirt uses quay.io for hosting images already and I've been
>> > looking at creating a script to automate usage of it via their
>> > REST API. Once done the same script should be usable by QEMU
>> > too.
>> >
>> > The idea would be that we still have docker files in the
>> > tests/docker/dockerfiles directory, but they would only be used
>> > for an automated job which triggers builds on quay.io, or for the
>> > few people who need to make changes to the dockerfiles. The current
>> > make rules used by developers / CI systems for executing test builds
>> > would be changed to simply pull the pre-built image off quay.io
>> > instead of running a docker build again.
>>
>> Could we just have a script that pulls the quay.io image and tags it as
>> the appropriate target and then we could do:
>>
>>   make docker-image-debian-arm64-cross [REBUILD=1]
>>
>> which would normally pull the quay.io image but if REBUILD=1 would force
>> a local rebuild?
>
> Perhaps, I hadn't really got as far as thinking about the make
> integration side.
>
>>
>> >> The other area that needs some work is better supporting non-x86 hosts.
>> >> While Docker's multi-arch story is much better (docker run debian:stable
>> >> will DTRT on any main architecture) we get stumped by things like
>> >> Debian's uneven support of cross compilers. For 4.1 I'd like to
>> >> reorganise the dockerfiles subdirectory into multiarch and arch specific
>> >> directories so we approach this is a less ad-hoc way. It would also be
>> >> nice to have the ability to gracefully fallback to linux-user powered
>> >> images where the host architecture doesn't have what we need.
>>
>> I suspect we'd never store linux-user powered images on quay.io as there
>> are niggly differences depending on the users binfmt_misc setup.
>
> Wouldn't we just upload the cross-build images and rely on the user's
> host to have registered binfmts needed to execute foreign binaries
> in the container ?

That's true - a linux-user powered image is really just a native arch
image with the addition of qemu. We already have the tooling to upload a
local QEMU into the image as-per the current binfmt_misc settings
(docker.py update).

--
Alex Bennée