Re: [PATCH 4/5] GitLab Gating CI: introduce pipeline-status contrib script

[Cleber Rosa]

On Thu, Jun 18, 2020 at 12:45:24PM +0100, Daniel P. Berrangé wrote:
> On Thu, Mar 12, 2020 at 03:36:15PM -0400, Cleber Rosa wrote:
> > This script is intended to be used right after a push to a branch.
> > 
> > By default, it will look for the pipeline associated with the commit
> > that is the HEAD of the *local* staging branch.  It can be used as a
> > one time check, or with the `--wait` option to wait until the pipeline
> > completes.
> > 
> > If the pipeline is successful, then a merge of the staging branch into
> > the master branch should be the next step.
> 
> On IRC yesterday we were discussing the current .gitlab-ci.yml status,
> and how frustrating it is that every time we get it green, more code is
> soon merged that turns it red again.
>

Hi Daniel,

I know this is nothing new to you, but "green" turning "red" is
something that can be minimized, but never completely abolished.
We've had discussions on how we could minimize those, and suggestions
on how to address those occurrences.  The points that I remember about
it are:

1. For jobs supposed to run on containers and vms, use images prepared
   to run the builds and tests, that is, don't do the prep steps
   (package installations and others) within the job itself - this is
   currently missing.

2. For bare metal, have the setup either documented or scripted and run
   before the jobs (in the patch series I sent, these were done in
   a number of Ansible based playbooks).

3. Have a mechanism for promoting non-gating jobs to gating jobs.  Keeping
   track of their reliability over a period of time is a way to forecast
   it future behavior, as much as it can be done.

4. Have a MAINTAINERS entry for those jobs, and have maintainers
   attemtp to address issues within a specified amount of time; if
   that fails, have a mechanism to downgrade those jobs to non-gating.
   A patch removing/moving entries from .gitlab-ci*.yml would do that
   and of course sping another set of CI jobs bringing things back to
   green.

> It feels like it should be an easy win to declare that the current GitLab
> CI jobs are to be used as a gating tests for merges to master.
>

I'm fine with that approach if most people agree.  This would mean
point #1 is ignored for now.  For everyone's sake, points #3 and #4
are a must IMO.  Point #2 will applied as we introduced our non-shared
gitlab runners.

> As & when custom runners come online, their jobs can simply augment the
> existing jobs. IOW, use of GitLab CI for gating master shouldn't be
> dependant on setup of custom runners which we've been waiting on for a
> long term.
> 
> Peter indicated that his main requirement is a way to automate the task
> of kicking off the CI job & getting its status. It seems like the script
> in this patch should fullfill that requirement.
> 
> Assuming Peter (or whomever is going todo the merge) has a fork of
> 
>    https://gitlab.com/qemu-project/qemu
> 
> then they need to find the "ID" number of their fork. This is
> visible at the top for the page for their fork eg mine:
> 
>    https://gitlab.com/berrange/qemu
> 
> reports:
> 
>    "Project ID: 18588805 "
> 
> Assuming the fork is configured as a git remote called "gitlab", then
> to use GitLab CI as gating test, all that appears to be needed is
> 
>    $ git push gitlab
>    $ ./contrib/ci/scripts/gitlab-pipeline-status --wait -p 18588805
> 
> If this is an acceptable level of automation for Peter, then can we
> get this specific patch merged right now and make current GitLab CI
> be gating for master.
> 
> 
> With GitLab as gating, then we have further incentive to move all
> the jobs currently on Travis CI and Shippable, over to use GitLab
> too, and also use cirrus-run  to make Cirrus CI jobs be proxied
> from GitLab.  All this can be in parallel with adding custom GitLab
> runners for expanding testing coverage still further.
>

The script should indeed work with the workflow you described.  I'll just
run a few tests, to make sure that are no caveats, and I'll let you and
Peter know about it.

Thanks,
- Cleber.

signature.asc
Description: PGP signature