[RFC PATCH] gitlab: add a binary build to project registry

[Alex Bennée]

There have been requests from other projects such LKFT to have
"official" docker images with pre-built QEMU binaries. These could
then be consumed by downstream CI systems by pulling directly from the
qemu-project container registry. The final image could then be run by
doing:

  docker run --rm -it \
    registry.gitlab.com/qemu-project/qemu/qemu/debian-amd64-binaries \
    /opt/bin/qemu-system-aarch64 $ARGS

To keep the build time down we skip user-mode, documents, plugins and
a selection of the more esoteric hardware emulations. Even so it still
takes over an hour to build and install everything.

Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Cc: Anders Roxell <anders.roxell@linaro.org>
Cc: Remi Duraffort <remi.duraffort@linaro.org>

---

This is very much an RFC for now as we have traditionally not provided
binaries for our users except for the slightly special case of
Windows. As currently structured this will rebuild the binaries on
every merge but we could change the generation rules to only trigger
for tagged or stable branch pushes. We also wouldn't be testing these
binaries so we are basically assuming they are OK by the nature of
being built from master which in theory never breaks.

The container layout is not ideal either as the QEMU dockerfiles are
all based on needing to build things so by their nature include all
the devel pacakges. We could get lcitool to spit out a non-dev version
with only binary dependencies but that would increase the number of
dockerfiles to manage. For now we just re-use the build images for the
final installed product.
---
 .gitlab-ci.d/binary-builds.yml                | 19 +++++++++++
 .../debian-amd64-binaries.docker              | 18 ++++++++++
 .gitlab-ci.d/container-template.yml           | 33 +++++++++++++++++++
 .gitlab-ci.d/qemu-project.yml                 |  1 +
 4 files changed, 71 insertions(+)
 create mode 100644 .gitlab-ci.d/binary-builds.yml
 create mode 100644 .gitlab-ci.d/binary-builds/debian-amd64-binaries.docker

diff --git a/.gitlab-ci.d/binary-builds.yml b/.gitlab-ci.d/binary-builds.yml
new file mode 100644
index 0000000000..bcb9d57f80
--- /dev/null
+++ b/.gitlab-ci.d/binary-builds.yml
@@ -0,0 +1,19 @@
+#
+# Binary Builds
+#
+# These are docker images with built QEMU binaries which could form
+# the basis of inputs for other projects that want recent QEMU builds
+# for their own CI.
+#
+
+include:
+  - local: '/.gitlab-ci.d/container-template.yml'
+
+build-binary-debian11:
+  extends: .container_native_template
+  needs:
+    - job: amd64-debian11-container
+      artifacts: true
+  timeout: 90m
+  variables:
+    NAME: debian-amd64-binaries
diff --git a/.gitlab-ci.d/binary-builds/debian-amd64-binaries.docker 
b/.gitlab-ci.d/binary-builds/debian-amd64-binaries.docker
new file mode 100644
index 0000000000..ea0c8fdf5b
--- /dev/null
+++ b/.gitlab-ci.d/binary-builds/debian-amd64-binaries.docker
@@ -0,0 +1,18 @@
+#
+# QEMU Binary Builds, in docker form
+#
+#
+
+# Build stage, assumes workdir
+FROM registry.gitlab.com/qemu-project/qemu/qemu/debian-amd64
+ADD . /src
+RUN mkdir -p /src/build
+WORKDIR /src/build
+RUN ../configure --disable-user --disable-plugins --disable-docs \
+  
--target-list-exclude=avr-softmmu,cris-softmmu,microblaze-softmmu,microblazeel-softmmu,or1k-softmmu,rx-softmmu,sh4-softmmu,sh4eb-softmmu,tricore-softmmu
 \
+  --prefix=/opt/qemu
+RUN ninja
+RUN make install
+
+FROM registry.gitlab.com/qemu-project/qemu/qemu/debian11
+COPY --from=0 /opt/qemu /opt/qemu
diff --git a/.gitlab-ci.d/container-template.yml 
b/.gitlab-ci.d/container-template.yml
index c434b9c8f3..be974d74ed 100644
--- a/.gitlab-ci.d/container-template.yml
+++ b/.gitlab-ci.d/container-template.yml
@@ -1,3 +1,8 @@
+#
+# This container job uses QEMU's wrapper scripts to build a container
+# which we can use for the basis of other tests. Generally these
+# containers setup build environments for building QEMU or its tests.
+#
 .container_job_template:
   extends: .base_job_template
   image: docker:stable
@@ -20,3 +25,31 @@
     - docker push "$TAG"
   after_script:
     - docker logout
+
+#
+# This container job uses the native docker tools to build and run and
+# container. Here the container may include the build itself which is
+# used for things like preparing binary images.
+#
+.container_native_template:
+  extends: .base_job_template
+  image: docker:stable
+  stage: containers
+  services:
+    - docker:dind
+  before_script:
+    - export TAG="$CI_REGISTRY_IMAGE/qemu/$NAME:latest"
+    - export COMMON_TAG="$CI_REGISTRY/qemu-project/qemu/$NAME:latest"
+    - apk add python3
+    - docker info
+    - docker login $CI_REGISTRY -u "$CI_REGISTRY_USER" -p 
"$CI_REGISTRY_PASSWORD"
+  script:
+    - echo "TAG:$TAG"
+    - echo "COMMON_TAG:$COMMON_TAG"
+    - DOCKER_BUILDKIT=1 docker build
+          -t "qemu/$NAME" -f ".gitlab-ci.d/binary-builds/$NAME.docker" .
+    - docker tag "qemu/$NAME" "$TAG"
+    - docker push "$TAG"
+  after_script:
+    - docker logout
+    
diff --git a/.gitlab-ci.d/qemu-project.yml b/.gitlab-ci.d/qemu-project.yml
index 691d9bf5dc..ed713f517a 100644
--- a/.gitlab-ci.d/qemu-project.yml
+++ b/.gitlab-ci.d/qemu-project.yml
@@ -13,3 +13,4 @@ include:
   - local: '/.gitlab-ci.d/custom-runners.yml'
   - local: '/.gitlab-ci.d/cirrus.yml'
   - local: '/.gitlab-ci.d/windows.yml'
+  - local: '/.gitlab-ci.d/binary-builds.yml'
-- 
2.30.2