[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [RFC PATCH v3 1/1] gitlab: Use plain docker in container-template.ym
From: |
Alex Bennée |
Subject: |
Re: [RFC PATCH v3 1/1] gitlab: Use plain docker in container-template.yml |
Date: |
Mon, 27 Feb 2023 19:39:11 +0000 |
User-agent: |
mu4e 1.9.21; emacs 29.0.60 |
Alex Bennée <alex.bennee@linaro.org> writes:
> Fabiano Rosas <farosas@suse.de> writes:
>
>> Our dockerfiles no longer reference layers from other qemu images so
>> we can now use 'docker build' on them.
>>
>> Also reinstate the caching that was disabled due to bad interactions
>> with certain runners. See commit 6ddc3dc7a8 ("tests/docker: don't use
>> BUILDKIT in GitLab either"). We now believe those issues to be fixed.
>>
>> The COMMON_TAG needed to be fixed for the caching to work. The
>> docker.py script was not using the variable, but constructing the
>> correct URL directly.
>>
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>> ---
>> .gitlab-ci.d/container-template.yml | 9 ++++-----
>> 1 file changed, 4 insertions(+), 5 deletions(-)
>>
>> diff --git a/.gitlab-ci.d/container-template.yml
>> b/.gitlab-ci.d/container-template.yml
>> index c434b9c8f3..519b8a9482 100644
>> --- a/.gitlab-ci.d/container-template.yml
>> +++ b/.gitlab-ci.d/container-template.yml
>> @@ -6,17 +6,16 @@
>> - docker:dind
>> before_script:
>> - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
>> - - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
>> + - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/qemu/$NAME:latest"
>> - apk add python3
>> - docker info
>> - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p
>> "$CI_REGISTRY_PASSWORD"
>> script:
>> - echo "TAG:$TAG"
>> - echo "COMMON_TAG:$COMMON_TAG"
>> - - ./tests/docker/docker.py --engine docker build
>> - -t "qemu/$NAME" -f "tests/docker/dockerfiles/$NAME.docker"
>> - -r $CI_REGISTRY/qemu-project/qemu
>> - - docker tag "qemu/$NAME" "$TAG"
>> + - docker build --tag "$TAG" --cache-from "$TAG" --cache-from
>> "$COMMON_TAG"
>> + --build-arg BUILDKIT_INLINE_CACHE=1
>> + -f "tests/docker/dockerfiles/$NAME.docker" "."
>
> I wonder why this doesn't injest a bunch of context. If I run:
>
> docker build --cache-from
> registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross --build-arg
> BUILDKIT_INLINE_CACHE=1 --build-arg USER=alex --build-arg UID=1000 -t
> qemu
> /debian-alpha-cross -f
> "/home/alex/lsrc/qemu.git/tests/docker/dockerfiles/debian-alpha-cross.docker"
> .
>
> it attempts to bring my entire build directory in as build context. This
> is why we use the - < docker form in the Makefile.
>
>> - docker push "$TAG"
>> after_script:
>> - docker logout
>
> So what I don't understand is if I do:
>
> docker pull registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross
> docker build --cache-from
> registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross --build-arg
> BUILDKIT_INLINE_CACHE=1 -t qemu/debian-alpha-cross - <
>
> /home/alex/lsrc/qemu.git/tests/docker/dockerfiles/debian-alpha-cross.docker
>
> I still see pretty much a full rebuild of the image.
For reference at the end:
🕙19:37:36 alex@zen:container-diff.git on master took 19s
➜ ./out/container-diff diff
daemon://registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross
daemon://qemu/debian-alpha-cross --type=file
-----File-----
These entries have been added to qemu/debian-alpha-cross:
FILE SIZE
/usr/share/doc/base-files/FAQ 6B
/usr/share/info/dir 781B
These entries have been deleted from qemu/debian-alpha-cross: None
These entries have been changed between
registry.gitlab.com/stsquad/qemu/qemu/debian-alpha-cross and
qemu/debian-alpha-cross:
FILE SIZE1 SIZE2
/var/cache/debconf/templates.dat 762.2K 764.1K
/var/cache/debconf/templates.dat-old 762.2K 764.1K
/var/lib/dpkg/available 70.7K 70.7K
/var/log/dpkg.log 12.9K 20.2K
/var/log/apt/term.log 7.4K 15.5K
/var/cache/ldconfig/aux-cache 5.5K 5.5K
/var/cache/debconf/config.dat-old 4.5K 4.7K
/var/log/apt/history.log 1.5K 2K
/etc/shadow 501B 501B
/etc/shadow- 501B 501B
/etc/apt/sources.list 430B 430B
--
Alex Bennée
Virtualisation Tech Lead @ Linaro