Re: RFC: docker for CI

[Janek Warchoł]

In principle, thumbs up. However, I think it's essential that we don't try
to do too much at once; I'd suggest to focus on one most important aspect
first. To do that, I'd like to ask a helper question: what are 2 most
important reasons for using Docker instead of Patchy? In other words, what
are 2 things that we want to be able to do which are impossible/difficult
with Patchy? (your proposal tells about the "what?" and the "how?", I'd
like to know the "why?")

cheers :-)
Janek



pt., 7 lut 2020 o 13:21 Han-Wen Nienhuys <address@hidden> napisał(a):

> Proposal: rather than using the patchy scripts for validating
> LilyPond, we use docker images.
>
> General idea
> ============
>
> There is a script ("driver") that drives docker running on a dedicated
> build machine ("host").
>
> There are several images:
>
> * The base dev image.
>
> The base image is based on some stripped Linux distribution, with all
> the devtools necessary for compiling LilyPond. In addition, it
> contains a copy of ccache, and a git clone of the LilyPond sourcecode
>
> * The base release image for a specific git commit.
>
> The procedure to build it is as follows:
>
>   * take the base dev image
>   * fetch the git commit
>   * runs (make ; make test-baseline)
>   * runs (make dist-clean)
>
> This saves the result as a docker image. The Docker image now contains
> a clean lilypond tree, the C++ compilation results (in ccache), and a
> test baseline.
>
> The base release image is made at official LilyPond releases, or at
> any release that has a new graphical regtest result
>
>
> CI: build binary
> ================
>
> Given a proposed change (as git commit):
>
>  * take base release image
>  * run (make; make doc) >& log-file
>
> On success, the driver saves the result as a docker image, tagged with the
> commit sha1.
>
> On failure, the driver uploads the last bit of the log-file to our code
> review system.
>
>
> CI: regtest
> ===========
>
> Given a proposed change (as git commit)
>
>   * take CI build image
>   * run (make check >& log-file)
>   * use a headless browser to take a image snapshot of the top of regtest
> result page.
>
>
> On success, the driver uploads the image snapshot to code review.
>
> On failure, the driver uploads the last bit of the log-file to code review.
>
>
> Considerations
> ==============
>
> * Because the build happens inside a container, we can test multiple
>   builds. We could build against guile 1.8 and 2.2 at the same time,
>   for example
>
> * Because the "build binary" step reuses CCache results, it can
>   complete quickly.
>
> * The regtest continues to be expensive to compute. In the future, I
>   hope it would not need a human to kick it off or post results back
>   into review, but likely, it should require a manual step in the
>   review process to kick off, eg. in Gerrit "Run-Regtest" +1 vote.
>
> * For security, the host should use https://github.com/google/gvisor
>   to avoid being hacked by malicious code in proposed changes.
>
> --
> Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen
>