Re: [PATCH 0/5] QEMU Gating CI

[Philippe Mathieu-Daudé]

On 4/23/20 7:13 PM, Daniel P. Berrangé wrote:
> On Thu, Apr 23, 2020 at 01:04:13PM -0400, Cleber Rosa wrote:
> > ----- Original Message -----
> > > From: "Peter Maydell" <address@hidden>
> > > To: "Markus Armbruster" <address@hidden>
> > > Cc: "Fam Zheng" <address@hidden>, "Thomas Huth" <address@hidden>, "Beraldo Leal" 
> > > <address@hidden>, "Erik
> > > Skultety" <address@hidden>, "Alex Bennée" <address@hidden>, "Wainer Moschetta" 
> > > <address@hidden>,
> > > "QEMU Developers" <address@hidden>, "Wainer dos Santos Moschetta" <address@hidden>, 
> > > "Willian Rampazzo"
> > > <address@hidden>, "Cleber Rosa" <address@hidden>, "Philippe Mathieu-Daudé" 
> > > <address@hidden>, "Eduardo
> > > Habkost" <address@hidden>
> > > Sent: Tuesday, April 21, 2020 8:53:49 AM
> > > Subject: Re: [PATCH 0/5] QEMU Gating CI
> > >
> > > On Thu, 19 Mar 2020 at 16:33, Markus Armbruster <address@hidden> wrote:
> > > > Peter Maydell <address@hidden> writes:
> > > > > I think we should start by getting the gitlab setup working
> > > > > for the basic "x86 configs" first. Then we can try adding
> > > > > a runner for s390 (that one's logistically easiest because
> > > > > it is a project machine, not one owned by me personally or
> > > > > by Linaro) once the basic framework is working, and expand
> > > > > from there.
> > > >
> > > > Makes sense to me.
> > > >
> > > > Next steps to get this off the ground:
> > > >
> > > > * Red Hat provides runner(s) for x86 stuff we care about.
> > > >
> > > > * If that doesn't cover 'basic "x86 configs" in your judgement, we
> > > >    fill the gaps as described below under "Expand from there".
> > > >
> > > > * Add an s390 runner using the project machine you mentioned.
> > > >
> > > > * Expand from there: identify the remaining gaps, map them to people /
> > > >    organizations interested in them, and solicit contributions from these
> > > >    guys.
> > > >
> > > > A note on contributions: we need both hardware and people.  By people I
> > > > mean maintainers for the infrastructure, the tools and all the runners.
> > > > Cleber & team are willing to serve for the infrastructure, the tools and
> > > > the Red Hat runners.
> > >
> > > So, with 5.0 nearly out the door it seems like a good time to check
> > > in on this thread again to ask where we are progress-wise with this.
> > > My impression is that this patchset provides most of the scripting
> > > and config side of the first step, so what we need is for RH to provide
> > > an x86 runner machine and tell the gitlab CI it exists. I appreciate
> > > that the whole coronavirus and working-from-home situation will have
> > > upended everybody's plans, especially when actual hardware might
> > > be involved, but how's it going ?
> >
> > Hi Peter,
> >
> > You hit the nail in the head here.  We were affected indeed with our ability
> > to move some machines from one lab to another (across the country), but we're
> > actively working on it.
>
> For x86, do we really need to be using custom runners ?
>
> With GitLab if someone forks the repo to their personal namespace, they
> cannot use any custom runners setup by the origin project. So if we use
> custom runners for x86, people forking won't be able to run the GitLab
> CI jobs.
>
> As a sub-system maintainer I wouldn't like this, because I ideally want
> to be able to run the same jobs on my staging tree, that Peter will run
> at merge time for the PULL request I send.
>
> Thus my strong preference would be to use the GitLab runners in every
> scenario where they are viable to use. Only use custom runners in the
> cases where GitLab runners are clearly inadequate for our needs.
>
> Based on what we've setup in GitLab for libvirt,  the shared runners
> they have work fine for x86. Just need the environments you are testing
> to be provided as Docker containers (you can actually build and cache
> the container images during your CI job too).  IOW, any Linux distro
> build and test jobs should be able to use shared runners on x86, and
> likewise mingw builds. Custom runners should only be needed if the
> jobs need todo *BSD / macOS builds, and/or have access to specific
> hardware devices for some reason.

Thanks to insist with that point Daniel. I'd rather see every 
configuration reproducible, so if we loose a hardware sponsor, we can 
find another one and start another runner.
Also note, if it is not easy to reproduce a runner, it will be very hard 
to debug a reported build/test error.

A non-reproducible runner can not be used as gating, because if they 
fail it is not acceptable to lock the project development process.


In some cases custom runners are acceptable. These runners won't be 
"gating" but can post informative log and status.

[*] Specific hardware that is not easily available.

- Alistair at last KVM forum talked about a RISCV board
  (to test host TCG)
- Aleksandar said at last KVM forum Wavecomp could plug a CI20 MIPS
  (to test host TCG)
- Lemote seems interested to setup some Loongson MIPSr6 board
  (to test interaction with KVM)

[*] To run code requiring accepting License Agreements

[*] To run non Free / Open Source code


Owner of these runners take the responsibility to provide enough 
time/information about reported bugs, or to debug them themselves.


Now the problem is GitLab runner is not natively available on the 
architectures listed in this mail, so custom setup is required. A dumb 
script running ssh to a machine also works (tested) but lot of manual 
tuning/maintenance expected.