[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [RFC PATCH 0/1] ci: Speed up container stage
From: |
Fabiano Rosas |
Subject: |
Re: [RFC PATCH 0/1] ci: Speed up container stage |
Date: |
Thu, 23 Feb 2023 17:30:58 -0300 |
Hi Alex,
> Daniel P. Berrangé <berrange@redhat.com> writes:
>
>> On Thu, Feb 23, 2023 at 11:21:53AM -0300, Fabiano Rosas wrote:
>>> I'm not sure if this was discussed previously, but I noticed we're not
>>> pulling the images we push to the registry at every pipeline run.
>>>
>>> I would expect we don't actually need to rebuild container images at
>>> _every_ pipeline run, so I propose we add a "docker pull" to the
>>> container templates. We already have that for the docker-edk2|opensbi
>>> images.
>>>
>>> Some containers can take a long time to build (14 mins) and pulling
>>> the image first without building can cut the time to about 3
>>> mins. With this we can save almost 2h of cumulative CI time per
>>> pipeline run:
>>
>> The docker.py script that we're invoking is already pulling the
>> image itself eg to pick a random recent job:
>>
>> https://gitlab.com/qemu-project/qemu/-/jobs/3806090058
>>
>> We can see
>>
>> $ ./tests/docker/docker.py --engine docker build -t "qemu/$NAME" -f
>> "tests/docker/dockerfiles/$NAME.docker" -r
>> $CI_REGISTRY/qemu-project/qemu 03:54
>> Using default tag: latest
>> latest: Pulling from qemu-project/qemu/qemu/debian-arm64-cross
>> bb263680fed1: Pulling fs layer
>> ...snip...
>>
>> none the less it still went ahead and rebuilt the image from scratch
>> so something is going wrong here. I don't know why your change adding
>> an extra 'docker pull' would have any effect, given we're already
>> pulling, so I wonder if that's just coincidental apparent change
>> due to the initial state of your fork's container registery.
>>
>> Whenever I look at this I end up wishing out docker.py didn't exist
>> and that we could just directly do
>>
>> - docker pull "$TAG"
>> - docker build --cache-from "$TAG" --tag "$TAG" -f
>> "tests/docker/$NAME.docker"
>>
>> as that sould be sufficient to build the image with caching.
>
> I think we should be ready to do that now as we have flattened all our
> dockerfiles. The only other thing that docker.py does is nicely add a
> final step for the current user so you can ensure all files generated in
> docker cross compile images are still readable on the host.
>
Just so you know this command line worked:
docker build --cache-from $TAG --tag $TAG --build-arg BUILDKIT_INLINE_CACHE=1 \
-f "tests/docker/dockerfiles/$NAME.docker" "."
building the cache: https://gitlab.com/farosas/qemu/-/jobs/3825838177
using the cache: https://gitlab.com/farosas/qemu/-/jobs/3825926944
But we might still have this issue:
commit 6ddc3dc7a882f2e7200fa7fecf505a8d0d8bbea9
Author: Daniel P. Berrangé <berrange@redhat.com>
Date: Fri Jul 9 15:29:35 2021 +0100
tests/docker: don't use BUILDKIT in GitLab either
Using BUILDKIT breaks with certain container registries such as CentOS,
with docker build reporting an error such as
failed to solve with frontend dockerfile.v0:
failed to build LLB: failed to load cache key:
unexpected status code
https://registry.centos.org/v2/centos/manifests/7:
403 Forbidden
We might need to go the route of skipping the docker build when the
docker pull succeeds.