Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines

[Cleber Rosa]

On Tue, Jun 8, 2021 at 2:27 PM Wainer dos Santos Moschetta
<wainersm@redhat.com> wrote:
>
> Hi,
>
> On 6/8/21 12:14 AM, Cleber Rosa wrote:
> > The QEMU project has two machines (aarch64 and s390x) that can be used
> > for jobs that do build and run tests.  This introduces those jobs,
> > which are a mapping of custom scripts used for the same purpose.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> >   .gitlab-ci.d/custom-runners.yml | 208 ++++++++++++++++++++++++++++++++
> >   1 file changed, 208 insertions(+)
> >
> > diff --git a/.gitlab-ci.d/custom-runners.yml 
> > b/.gitlab-ci.d/custom-runners.yml
> > index a07b27384c..061d3cdfed 100644
> > --- a/.gitlab-ci.d/custom-runners.yml
> > +++ b/.gitlab-ci.d/custom-runners.yml
> > @@ -12,3 +12,211 @@
> >   # guarantees a fresh repository on each job run.
> >   variables:
> >     GIT_STRATEGY: clone
> > +
> > +# All ubuntu-18.04 jobs should run successfully in an environment
> > +# setup by the scripts/ci/setup/build-environment.yml task
> > +# "Install basic packages to build QEMU on Ubuntu 18.04/20.04"
> > +ubuntu-18.04-s390x-all-linux-static:
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
>
> Should it restrict the job for pushes to qemu-project only? If yes, then
> it probably needs the statement:
>
> '$CI_PROJECT_NAMESPACE == "qemu-project" && $CI_COMMIT_BRANCH =~ /^staging/'
>

I'm not sure we should.  In theory, if people have access to other
machines on their own accounts, they should be able to trigger the
same jobs just by using the "staging" prefix.

> If you change that here, you will end it changing all over the jobs. In
> general, there are many boilerplates in this file. I'm ok to merge it as
> is as long as it is followed by another series to refactor the code.
>

Absolutely, as mentioned before, this is a straightforward mapping of
Peter's jobs, so I don't want to go over too many inderaction and
abstraction levels initially.

> > + script:
> > + # --disable-libssh is needed because of 
> > https://bugs.launchpad.net/qemu/+bug/1838763
> > + # --disable-glusterfs is needed because there's no static version of 
> > those libs in distro supplied packages
> > + - mkdir build
> > + - cd build
> > + - ../configure --enable-debug --static --disable-system 
> > --disable-glusterfs --disable-libssh
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > + - make --output-sync -j`nproc` check-tcg V=1
> > +
> > +ubuntu-18.04-s390x-all:
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> > + script:
> > + - mkdir build
> > + - cd build
> > + - ../configure --disable-libssh
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > +
> > +ubuntu-18.04-s390x-alldbg:
> Maybe we don't need both ubuntu-18.04-s390x-all and
> ubuntu-18.04-s390x-alldbg jobs.
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> > + script:
> > + - mkdir build
> > + - cd build
> > + - ../configure --enable-debug --disable-libssh
> > + - make clean
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > +ubuntu-18.04-s390x-clang:
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> > +   when: manual
> > + script:
> > + - mkdir build
> > + - cd build
> > + - ../configure --disable-libssh --cc=clang --cxx=clang++ 
> > --enable-sanitizers
> > + - make --output-sync -j`nproc`
> > + - make --output-sync -j`nproc` check V=1
> > +
> > +ubuntu-18.04-s390x-tci:
> > + allow_failure: true
> > + needs: []
> > + stage: build
> > + tags:
> > + - ubuntu_18.04
> > + - s390x
> > + rules:
> > + - if: '$CI_COMMIT_BRANCH =~ /^staging/'
> > + script:
> > + - mkdir build
> > + - cd build
> > + - ../configure --disable-libssh --enable-tcg-interpreter
> > + - make --output-sync -j`nproc`
> I think it needs to `make check-tcg` at least. See "build-tci" in
> `.gitlab-ci.d/buildtest.yml` for other tests being executed on shared
> runners.
> > +
> > +ubuntu-18.04-s390x-notcg:
> The "build-tcg-disabled" in `.gitlab-ci.d/buildtest.yml` could be
> mimic-ed here too.

These are straightforward mappings of Peter's jobs... I honestly don't
think we should deviate any further at this time.  Let me know if you
think I'm missing something though.

Thanks for the review!
- Cleber.