Re: [PATCH 0/7] testing/next: docker.py removal and kaniko updates

[Daniel P . Berrangé]

On Tue, Feb 28, 2023 at 02:29:12PM +0100, Philippe Mathieu-Daudé wrote:
> On 28/2/23 13:58, Daniel P. Berrangé wrote:
> > On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote:
> > > On 24/2/23 19:08, Alex Bennée wrote:
> > > > This series attempts to remove our dependence on the docker.py script
> > > > and build things directly with the appropriate tool. I've been
> > > > noodling around with how we build images on gitlab to see if they can
> > > > cache better because the normal case should be we don't need to
> > > > rebuild everything if the upstream distro hasn't updated its package
> > > > list.
> > > > 
> > > > Anyway what do people think?
> > > 
> > > Removing dind limitation is interesting.
> > > 
> > > Unrelated, can we tag registry.gitlab.com/qemu-project's
> > > docker images along with releases?
> > 
> > Can you elaborate on this ?
> > 
> > We're only using the images for CI purposes and they must always reflect
> > the current state of git master. We're using a fixed docker tag 'latest',
> > as that avoids the container registry growing arbitrarily large.
> > 
> > Our CI rules should prevent the pipelines running on stable branches,
> > so we shouldn't need container tags for stable.
> 
> I'm not suggesting keeping jobs to build, but doing a snapshot of the
> released containers.
> 
> I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch
> it again. This is useful when bisecting pre-v8, but also to build pre-v8
> and do performance comparison. One shouldn't have to upgrade such
> container (in particular when package mirror disappear), since it
> already contains all we need.

The main risk with this is the impact on our storage quota. With the
OSS Program membership IIUC we get Ultimate level features which
is 250 GB of storage, across git repos, pipeline cache/artifacts/logs,
container registry.

Currently they have no way to enforce that since their accounting of
usage is not accurate enough. They're working on fixing that so at
somepoint we'll be subject to the 250 GB limit.

What I don't know is how much storage we're currently using across
the /qemu-project namespace, and what extra is implied by taking
a snapshot of our container registry 3 times a year. I'm expecting
it to probably be in the high 10's of GB though for the container
registry.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|