Re: continuous integration

[Tim Rühsen]

On 26.03.20 00:46, Bruno Haible wrote:
> Jeffrey Walton wrote:
>> CI tests should be catching these mistakes. (And problems like
>> _NoReturn on OS X).
> 
> Yes, CI can catch some mistakes. Like, just last week, this one: [1].
> 
> Tim and I maintain a continuous integration for gnulib at [2].
> 
> More effort could be put in, in two directions:
> 
> * Like Paul says, instead of only building testdirs, it could build
>   some packages that use gnulib. I would estimate that this would catch
>   3x as many bugs as the current CI with just testdirs.
> 
> * Like you suggest, it would also be useful to test macOS, FreeBSD,
>   Cygwin, and mingw builds.
> 
>> Is there any reasons services like Travis or Cirrus are not being used
>> to proactively detect problems on Linux, OS X and FreeBSD?
> 
> For my part:
> 
> * I have only limited time to work on this; that's why I limit
>   myself to CI integrations for a couple of packages on gitlab.

Same here. I really wish we could had more time to put into CI runners.

I would like to point out that debugging using a CI like Travis is
absolutely tedious and might take a lot of time. Docker-based CI (Linux
only :-| ) are so much easier to debug as you can run the test
environment (images + build scripts) locally.

So while some errors are obvious and easy to fix, others are a nightmare
as you can't 'log in' and just use a debugger. At least I don't have VMs
with OSX or Windows for this purpose.

Did anyone think about using the gcc build platform for automated
testing ? I made up some scripts a while ago for Wget but then lost
focus... if someone likes to take that up.

> * I had not heard of Cirrus CI. Coverage of FreeBSD, additionally to
>   Windows and macOS, sounds interesting. [3]
> 
> * Travis and Cirrus CI are most easily used on Github [4][5]. I don't
>   much like to work on Github, because it tends to become a closed
>   environment. E.g.
>     - You can fork someone else's repository only if you stay on Github.
>     - Many developers' email addresses are not published, which prevents
>       you from reporting issues by email. You have to use Github "issues"
>       instead.

We just need a mirror / fork on Github that we push to (sync) from time
to time. If someone cares for the initial Travis and/or Cirrus setup
with OSX / FreeBSD / Windows in mind, that would be great !

> But if someone wants to set it up and maintain it, I'm all for it!
> 
> Bruno
> 
> [1] https://lists.gnu.org/archive/html/bug-gnulib/2020-03/msg00041.html
> [2] https://gitlab.com/gnulib/gnulib-ci
> [3] https://cirrus-ci.org/features/#comparison-with-popular-ciaas
> [4] https://en.wikipedia.org/wiki/Travis_CI
> [5] https://cirrus-ci.org/faq/#only-github-support
> 
> 

Regards, Tim

signature.asc
Description: OpenPGP digital signature