Re: [PATCH 2/2] gitlab: don't run CI jobs by default on push to user forks

[Cornelia Huck]

On Mon, Aug 16 2021, Daniel P. Berrangé <berrange@redhat.com> wrote:

> On Mon, Aug 16, 2021 at 01:47:08PM +0200, Cornelia Huck wrote:
>> On Mon, Aug 16 2021, Daniel P. Berrangé <berrange@redhat.com> wrote:
>> 
>> > When I'm working on changing gitlab CI rules, then I burn loads of
>> > minutes which is especially troubling - limited CI minutes will make
>> > it very hard for me to debug future CI rule changes :-(
>> 
>> I hope that will not make gitlab CI a complete non-starter -- if you
>> cannot easily debug a test case that is failing, it's mostly
>> useless. We've seen too many cases where a failure could not be
>> reproduced when the test case was running locally.
>
> One of the best things about GitLab compared to what we had with
> Travis is that the build environment is 100% container based (until
> Cleber's custom runners arrived).  This meant that when something
> does fail in GitLab, you can pull the container image down from
> the gitlab registry and run the build locally. You still have
> differences due to hardware or CPUs resources, but its a hell of
> alot easier to reproduce than before. This is good enough for most
> contributors in general, but not for the CI maintainers, who need
> to debug especially the CI system as it exists on GitLab.

Correct me if I'm wrong, but I remember that some of the more
aggravating failures in particular could not be reproduced outside of
the gitlab environment, even while using the same image.

>
>
>> >> (c) a subsystem maintainer is preparing a pull request
>> >> 
>> >> Ideally, that should run the 'gating' set, to eliminate needless bounces
>> >> of the pull request; plus some subsystem-specific manual testing on
>> >> top. In practice, the 'full' set might be good enough.
>> >
>> > Yeah, the full/gating set is what I would thing subsys maintainers
>> > would want to use, to minimize risk that Peter's tests throw back
>> > the merge due to failure. The only difference of gating vs full
>> > is whether the acceptance tests run.
>> 
>> I can at least run a subset of the acceptance tests locally, but I think
>> I may be missing some platforms? Still, better than nothing.
>
> As ever the big problem for most people is non-x86_64 platforms. The
> custom runners solve this for gitlab, and in theory people can deploy
> a VM using QEMU TCG to do this locally, but massively slower

Still, the acceptance tests are usually small enough that running under
tcg is bearable.