[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 2/3] gitlab: support for FreeBSD 12, 13 and macOS 11 via cirr
|
From: |
Thomas Huth |
|
Subject: |
Re: [PATCH 2/3] gitlab: support for FreeBSD 12, 13 and macOS 11 via cirrus-run |
|
Date: |
Mon, 28 Jun 2021 09:28:18 +0200 |
|
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 |
On 25/06/2021 19.22, Daniel P. Berrangé wrote:
This adds support for running 4 jobs via Cirrus CI runners:
* FreeBSD 12
* FreeBSD 13
* macOS 11 with default XCode
* macOS 11 with latest XCode
The gitlab job uses a container published by the libvirt-ci
project (https://gitlab.com/libvirt/libvirt-ci) that contains
the 'cirrus-run' command. This accepts a short yaml file that
describes a single Cirrus CI job, runs it using the Cirrus CI
REST API, and reports any output to the console.
In this way Cirrus CI is effectively working as an indirect
custom runner for GitLab CI pipelines. The key benefit is that
Cirrus CI job results affect the GitLab CI pipeline result and
so the user only has look at one CI dashboard.
Cool, thanks for tackling this!
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
.gitlab-ci.d/cirrus.yml | 103 ++++++++++++++++++++++++++++
.gitlab-ci.d/cirrus/README.rst | 54 +++++++++++++++
.gitlab-ci.d/cirrus/build.yml | 35 ++++++++++
.gitlab-ci.d/cirrus/freebsd-12.vars | 13 ++++
.gitlab-ci.d/cirrus/freebsd-13.vars | 13 ++++
.gitlab-ci.d/cirrus/macos-11.vars | 15 ++++
.gitlab-ci.d/qemu-project.yml | 1 +
7 files changed, 234 insertions(+)
create mode 100644 .gitlab-ci.d/cirrus.yml
create mode 100644 .gitlab-ci.d/cirrus/README.rst
create mode 100644 .gitlab-ci.d/cirrus/build.yml
create mode 100644 .gitlab-ci.d/cirrus/freebsd-12.vars
create mode 100644 .gitlab-ci.d/cirrus/freebsd-13.vars
create mode 100644 .gitlab-ci.d/cirrus/macos-11.vars
diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml
new file mode 100644
index 0000000000..d7b4cce79b
--- /dev/null
+++ b/.gitlab-ci.d/cirrus.yml
@@ -0,0 +1,103 @@
+# Jobs that we delegate to Cirrus CI because they require an operating
+# system other than Linux. These jobs will only run if the required
+# setup has been performed on the GitLab account.
+#
+# The Cirrus CI configuration is generated by replacing target-specific
+# variables in a generic template: some of these variables are provided
+# when the GitLab CI job is defined, others are taken from a shell
+# snippet generated using lcitool.
+#
+# Note that the $PATH environment variable has to be treated with
+# special care, because we can't just override it at the GitLab CI job
+# definition level or we risk breaking it completely.
+.cirrus_build_job:
+ stage: build
+ image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master
+ needs: []
+ script:
+ - source .gitlab-ci.d/cirrus/$NAME.vars
+ - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g"
+ -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g"
+ -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g"
+ -e "s|[@]CIRRUS_VM_INSTANCE_TYPE@|$CIRRUS_VM_INSTANCE_TYPE|g"
+ -e "s|[@]CIRRUS_VM_IMAGE_SELECTOR@|$CIRRUS_VM_IMAGE_SELECTOR|g"
+ -e "s|[@]CIRRUS_VM_IMAGE_NAME@|$CIRRUS_VM_IMAGE_NAME|g"
+ -e "s|[@]CIRRUS_VM_CPUS@|$CIRRUS_VM_CPUS|g"
+ -e "s|[@]CIRRUS_VM_RAM@|$CIRRUS_VM_RAM|g"
+ -e "s|[@]UPDATE_COMMAND@|$UPDATE_COMMAND|g"
+ -e "s|[@]INSTALL_COMMAND@|$INSTALL_COMMAND|g"
+ -e "s|[@]PATH@|$PATH_EXTRA${PATH_EXTRA:+:}\$PATH|g"
+ -e "s|[@]PKG_CONFIG_PATH@|$PKG_CONFIG_PATH|g"
+ -e "s|[@]PKGS@|$PKGS|g"
+ -e "s|[@]MAKE@|$MAKE|g"
+ -e "s|[@]PYTHON@|$PYTHON|g"
+ -e "s|[@]PIP3@|$PIP3|g"
+ -e "s|[@]PYPI_PKGS@|$PYPI_PKGS|g"
+ -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g"
+ -e "s|[@]TEST_TARGETSS@|$TEST_TARGETSS|g"
+ <.gitlab-ci.d/cirrus/build.yml >.gitlab-ci.d/cirrus/$NAME.yml
+ - cat .gitlab-ci.d/cirrus/$NAME.yml
+ - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml
+ rules:
+ - if: "$TEMPORARILY_DISABLED"
+ allow_failure: true
+ - if: "$CIRRUS_GITHUB_REPO && $CIRRUS_API_TOKEN"
+
+x64-freebsd-12-build:
+ extends: .cirrus_build_job
+ variables:
+ NAME: freebsd-12
+ CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
+ CIRRUS_VM_IMAGE_SELECTOR: image_family
+ CIRRUS_VM_IMAGE_NAME: freebsd-12-2
+ CIRRUS_VM_CPUS: 8
+ CIRRUS_VM_RAM: 8G
+ UPDATE_COMMAND: pkg update
+ INSTALL_COMMAND: pkg install -y
+ # TODO: Enable gnutls again once FreeBSD's libtasn1 got fixed
+ # See: https://gitlab.com/gnutls/libtasn1/-/merge_requests/71
+ CONFIGURE_ARGS: --disable-gnutls
+ TEST_TARGETS: check
+
+x64-freebsd-13-build:
+ extends: .cirrus_build_job
+ variables:
+ NAME: freebsd-13
+ CIRRUS_VM_INSTANCE_TYPE: freebsd_instance
+ CIRRUS_VM_IMAGE_SELECTOR: image_family
+ CIRRUS_VM_IMAGE_NAME: freebsd-13-0
+ CIRRUS_VM_CPUS: 8
+ CIRRUS_VM_RAM: 8G
+ UPDATE_COMMAND: pkg update
+ INSTALL_COMMAND: pkg install -y
+ TEST_TARGETS: check
+
+x64-macos-11-base-build:
+ extends: .cirrus_build_job
+ variables:
+ NAME: macos-11
+ CIRRUS_VM_INSTANCE_TYPE: osx_instance
+ CIRRUS_VM_IMAGE_SELECTOR: image
+ CIRRUS_VM_IMAGE_NAME: big-sur-base
+ CIRRUS_VM_CPUS: 12
+ CIRRUS_VM_RAM: 24G
+ UPDATE_COMMAND: brew update
+ INSTALL_COMMAND: brew install
+ PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin
+ PKG_CONFIG_PATH:
/usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
+ TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat
check-qtest-x86_64
+
+x64-macos-11-xcode-build:
+ extends: .cirrus_build_job
+ variables:
+ NAME: macos-11
+ CIRRUS_VM_INSTANCE_TYPE: osx_instance
+ CIRRUS_VM_IMAGE_SELECTOR: image
+ CIRRUS_VM_IMAGE_NAME: big-sur-xcode
+ CIRRUS_VM_CPUS: 12
+ CIRRUS_VM_RAM: 24G
+ UPDATE_COMMAND: brew update
+ INSTALL_COMMAND: brew install
+ PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin
+ PKG_CONFIG_PATH:
/usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig
+ TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat
check-qtest-x86_64
Not directly related to your patch, but I wonder whether we really gain much
by having two macos build jobs in our CI ... they seem to be very similar,
e.g. compare the output of the "configure" step of the "macos" and the
"macos_xcode" job here:
https://cirrus-ci.com/build/4919300914937856
The diff basically looks like this:
diff -u /tmp/macos.txt /tmp/macos_xcode.txt
--- /tmp/macos.txt 2021-06-28 09:18:48.218698482 +0200
+++ /tmp/macos_xcode.txt 2021-06-28 09:21:01.811348076 +0200
@@ -1,6 +1,6 @@
mkdir build
cd build
-../configure --python=/usr/local/bin/python3 --enable-werror
--extra-cflags='-Wno-error=deprecated-declarations' || { cat config.log
meson-logs/meson-log.txt; exit 1; }
+../configure --extra-cflags='-Wno-error=deprecated-declarations'
--enable-modules --enable-werror --cc=clang || { cat config.log
meson-logs/meson-log.txt; exit 1; }
Disabling PIE due to missing toolchain support
Submodule 'capstone' (https://gitlab.com/qemu-project/capstone.git)
registered for path 'capstone'
Submodule 'dtc' (https://gitlab.com/qemu-project/dtc.git) registered for
path 'dtc'
@@ -23,18 +23,18 @@
Build type: native build
Project name: qemu
Project version: 6.0.50
-C compiler for the host machine: cc (clang 12.0.0 "Apple clang version
12.0.0 (clang-1200.0.32.28)")
-C linker for the host machine: cc ld64 609.8
+C compiler for the host machine: clang (clang 12.0.0 "Apple clang version
12.0.0 (clang-1200.0.32.27)")
+C linker for the host machine: clang ld64 609.7
Host machine cpu family: x86_64
Host machine cpu: x86_64
../meson.build:10: WARNING: Module unstable-keyval has no backwards or
forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/local/opt/python@3.9/bin/python3.9)
Program bzip2 found: YES
-C++ compiler for the host machine: c++ (clang 12.0.0 "Apple clang version
12.0.0 (clang-1200.0.32.28)")
-C++ linker for the host machine: c++ ld64 609.8
+C++ compiler for the host machine: c++ (clang 12.0.0 "Apple clang version
12.0.0 (clang-1200.0.32.27)")
+C++ linker for the host machine: c++ ld64 609.7
Objective-C compiler for the host machine: clang (clang 12.0.0)
-Objective-C linker for the host machine: clang ld64 609.8
+Objective-C linker for the host machine: clang ld64 609.7
Program cgcc found: NO
Library m found: YES
Library util found: YES
@@ -238,7 +238,7 @@
Program diff found: YES
Program dbus-daemon found: NO
Program initrd-stress.sh found: YES
-Build targets in project: 534
+Build targets in project: 556
qemu 6.0.50
@@ -273,7 +273,8 @@
user-mode emulation: NO
block layer: YES
Install blobs: YES
- module support: NO
+ module support: YES
+ alternative module path: NO
plugin support: NO
fuzzing support: NO
Audio drivers: coreaudio sdl
@@ -293,8 +294,8 @@
Compilation
host CPU: x86_64
host endianness: little
- C compiler: cc
- Host C compiler: cc
+ C compiler: clang
+ Host C compiler: clang
C++ compiler: c++
Objective-C compiler: clang
ARFLAGS: rv
@@ -376,7 +377,7 @@
pixman: YES
VTE support: NO
slirp support: internal
- libtasn1: NO
+ libtasn1: YES
PAM: YES
iconv support: YES
curses support: NO
@@ -403,7 +404,7 @@
xfsctl support: NO
smartcard support: NO
U2F support: NO
- libusb: NO
+ libusb: YES
usb net redir: NO
OpenGL support: NO
GBM: NO
... IMHO it's not worth wasting precious CI minutes here just because of
those small differences, and I'd like to suggest that we continue with only
on macos job in the future.
diff --git a/.gitlab-ci.d/cirrus/README.rst b/.gitlab-ci.d/cirrus/README.rst
new file mode 100644
index 0000000000..657b0706d7
--- /dev/null
+++ b/.gitlab-ci.d/cirrus/README.rst
@@ -0,0 +1,54 @@
+Cirrus CI integration
+=====================
+
+GitLab CI shared runners only provide a docker environment running on Linux.
+While it is possible to provide private runners for non-Linux platforms this
+is not something most contributors/maintainers will wish to do.
+
+To work around this limitation, we take advantage of `Cirrus CI`_'s free
+offering: more specifically, we use the `cirrus-run`_ script to trigger Cirrus
+CI jobs from GitLab CI jobs so that Cirrus CI job output is integrated into
+the main GitLab CI pipeline dashboard.
+
+There is, however, some one-time setup required. If you want FreeBSD and macOS
+builds to happen when you push to your GitLab repository, you need to
+
+* set up a GitHub repository for the project, eg. ``yourusername/qemu``.
+ This repository needs to exist for cirrus-run to work, but it doesn't need to
+ be kept up to date, so you can create it and then forget about it;
+
+* enable the `Cirrus CI GitHub app`_ for your GitHub account;
+
+* sign up for Cirrus CI. It's enough to log into the website using your GitHub
+ account;
+
+* grab an API token from the `Cirrus CI settings`_ page;
+
+* it may be necessary to push an empty ``.cirrus.yml`` file to your github fork
+ for Cirrus CI to properly recognize the project. You can check whether
+ Cirrus CI knows about your project by navigating to:
+
+ ``https://cirrus-ci.com/yourusername/qemu``
+
+* in the *CI/CD / Variables* section of the settings page for your GitLab
+ repository, create two new variables:
+
+ * ``CIRRUS_GITHUB_REPO``, containing the name of the GitHub repository
+ created earlier, eg. ``yourusername/qemu``;
+
+ * ``CIRRUS_API_TOKEN``, containing the Cirrus CI API token generated earlier.
+ This variable **must** be marked as *Masked*, because anyone with knowledge
+ of it can impersonate you as far as Cirrus CI is concerned.
+
+ Neither of these variables should be marked as *Protected*, because in
+ general you'll want to be able to trigger Cirrus CI builds from non-protected
+ branches.
+
+Once this one-time setup is complete, you can just keep pushing to your GitLab
+repository as usual and you'll automatically get the additional CI coverage.
+
+
+.. _Cirrus CI GitHub app: https://github.com/marketplace/cirrus-ci
+.. _Cirrus CI settings: https://cirrus-ci.com/settings/profile/
+.. _Cirrus CI: https://cirrus-ci.com/
+.. _cirrus-run: https://github.com/sio/cirrus-run/
diff --git a/.gitlab-ci.d/cirrus/build.yml b/.gitlab-ci.d/cirrus/build.yml
new file mode 100644
index 0000000000..857bdc5536
--- /dev/null
+++ b/.gitlab-ci.d/cirrus/build.yml
@@ -0,0 +1,35 @@
+@CIRRUS_VM_INSTANCE_TYPE@:
+ @CIRRUS_VM_IMAGE_SELECTOR@: @CIRRUS_VM_IMAGE_NAME@
+ cpu: @CIRRUS_VM_CPUS@
+ memory: @CIRRUS_VM_RAM@
+
+env:
+ CIRRUS_CLONE_DEPTH: 1
+ CI_REPOSITORY_URL: "@CI_REPOSITORY_URL@"
+ CI_COMMIT_REF_NAME: "@CI_COMMIT_REF_NAME@"
+ CI_COMMIT_SHA: "@CI_COMMIT_SHA@"
+ PATH: "@PATH@"
+ PKG_CONFIG_PATH: "@PKG_CONFIG_PATH@"
+ PYTHON: "@PYTHON@"
+ MAKE: "@MAKE@"
+ CONFIGURE_ARGS: "@CONFIGURE_ARGS@"
+
+build_task:
+ install_script:
+ - @UPDATE_COMMAND@
+ - @INSTALL_COMMAND@ @PKGS@
+ - if test -n "@PYPI_PKGS@" ; then @PIP3@ install @PYPI_PKGS@ ; fi
+ clone_script:
+ - git clone --depth 100 "$CI_REPOSITORY_URL" .
+ - git fetch origin "$CI_COMMIT_REF_NAME"
+ - git reset --hard "$CI_COMMIT_SHA"
+ build_script:
+ - mkdir build
+ - cd build
+ - ../configure --enable-werror $CONFIGURE_ARGS
+ || { cat config.log meson-logs/meson-log.txt; exit 1; }
+ - $MAKE -j$(sysctl -n hw.ncpu)
+ - for TARGET in $TEST_TARGETS ;
+ do
+ $MAKE -j$(sysctl -n hw.ncpu) $TARGET V=1 ;
+ done
You seem to try to enable ccache in the other files (e.g. by extending the
PATH) ... however, I don't see where you try to save the ccache directory
between the runs... so I guess that ccache won't be working with this setup
yet? In that case, I'd recommend to rather drop the other ccache changes
again, since the initial run with ccache (where the cache gets populated) is
rather slower than compiling without ccache.
Thomas