On Fri, Feb 17, 2023 at 11:35:44AM -0500, John Snow wrote:
> On Thu, Feb 16, 2023, 2:44 PM Daniel P. Berrangé <berrange@redhat.com>
> wrote:
>
> > On Thu, Feb 16, 2023 at 01:15:30PM -0500, John Snow wrote:
> > > On Wed, Feb 15, 2023 at 2:25 PM Alex Bennée <alex.bennee@linaro.org>
> > wrote:
> > > >
> > > > The 22.04 LTS release has been out for almost a year now so its time
> > > > to update all the remaining images to the current LTS. We can also
> > > > drop some hacks we need for older clang TSAN support.
> > >
> > > We still support Ubuntu 20.04 until 2024 though, don't we? Is it safe
> > > to not test this platform?
> > >
> > > I've long been uncertain about what our policy actually is for docker
> > > tests, if we want to test every platform we support or only some of
> > > them; and if it's only some of them, when do we choose the older and
> > > when do we choose the newer?
> >
> > Ideally we would test both the oldest & newest versions of each
> > distro we support. Practically though, we're compromised by the
> > limited CI resources available.
> >
>
> Yes, understood.
>
>
> > Dropping older Ubuntu images is a reasonable tradeoff, since we
> > still have Debian images covered in CI. Debian can be thought
> > of as an older version of Ubuntu to some extent, giving coverage
> > that will mitigate the risks of dropping 20.04.
> >
>
> Okay, I'll take your word for that. I am not personally familiar with how
> much those distros diverge; I know Ubuntu is debian-based but that's the
> extent of my knowledge as I don't daily-drive either.
>
> So, firstly:
>
> Reviewed-by: John Snow <jsnow@redhat.com>
>
> because I suspect we all have our reasons and I also agree testing newer is
> generally of higher value than testing older.
>
> However, would it be possible to keep the older Ubuntu test as a manual
> execution that we could invoke at will, only during RC testing phase? If
> it's not a lot of work, I could even check that in myself as a follow-up if
> it isn't unwanted.
>
> I find that "oldest version of x" is quite useful to me for testing Python
> stuff in particular, as that ecosystem moves pretty fast. It'd be mighty
> convenient to me in particular to keep an old Ubuntu test around to run
> manually as needed.
>
> (Heck, even if it wasn't on CI at all but was just a container I could run
> locally, that would still be quite useful.)
>
> Whaddaya think?
It would be pretty trivial to have tests/docker/dockerfiles contain
Dockerfiles for *every* supported distro version we have, and then
only build & test a subset in CI. It would merely suggest that we
change our naming convention so the dockerfiles in that dir include
the version. Basically adopting the standard libvirt-ci naming
convention for targets of $OSNAME-$OSVERSION:
$ lcitool targets
almalinux-8
almalinux-9
alpine-315
alpine-316
alpine-edge
centos-stream-8
centos-stream-9
debian-10
debian-11
debian-sid
fedora-36
fedora-37
fedora-rawhide
freebsd-12
freebsd-13
freebsd-current
macos-12
macos-13
Wait, what? How does this work??
opensuse-leap-153
opensuse-leap-154
opensuse-tumbleweed
ubuntu-1804
ubuntu-2004
ubuntu-2204
Contributors can then use 'make docker-XXXX' to run build tests
locally on specific distros when they need to test something
that isn't covered by default in out gating CI
OK, I might follow up on this, then. I would find this useful for proving certain python build system changes are not disruptive.
In contrast to C world, I find modern Pythonisms sneak in with quite an increased frequency, so having manual testing for the oldest platforms has some value there, but only every once in a while. Not worth our CI minutes.
Carry on as normal for this series, please and thank you!