[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[PULL v5 02/44] Jobs based on custom runners: build environment docs and
From: |
Alex Bennée |
Subject: |
[PULL v5 02/44] Jobs based on custom runners: build environment docs and playbook |
Date: |
Wed, 14 Jul 2021 15:59:54 +0100 |
From: Cleber Rosa <crosa@redhat.com>
To run basic jobs on custom runners, the environment needs to be
properly set up. The most common requirement is having the right
packages installed.
The playbook introduced here covers the QEMU's project s390x and
aarch64 machines. At the time this is being proposed, those machines
have already had this playbook applied to them.
Signed-off-by: Cleber Rosa <crosa@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Alex Bennée <alex.bennee@linaro.org>
Tested-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Wainer dos Santos Moschetta <wainersm@redhat.com>
Reviewed-by: Willian Rampazzo <willianr@redhat.com>
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20210630012619.115262-3-crosa@redhat.com>
Message-Id: <20210709143005.1554-3-alex.bennee@linaro.org>
diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
index 064ffa9988..bfedbb1025 100644
--- a/docs/devel/ci.rst
+++ b/docs/devel/ci.rst
@@ -30,3 +30,43 @@ The GitLab CI jobs definition for the custom runners are
located under::
Custom runners entail custom machines. To see a list of the machines
currently deployed in the QEMU GitLab CI and their maintainers, please
refer to the QEMU `wiki <https://wiki.qemu.org/AdminContacts>`__.
+
+Machine Setup Howto
+-------------------
+
+For all Linux based systems, the setup can be mostly automated by the
+execution of two Ansible playbooks. Create an ``inventory`` file
+under ``scripts/ci/setup``, such as this::
+
+ fully.qualified.domain
+ other.machine.hostname
+
+You may need to set some variables in the inventory file itself. One
+very common need is to tell Ansible to use a Python 3 interpreter on
+those hosts. This would look like::
+
+ fully.qualified.domain ansible_python_interpreter=/usr/bin/python3
+ other.machine.hostname ansible_python_interpreter=/usr/bin/python3
+
+Build environment
+~~~~~~~~~~~~~~~~~
+
+The ``scripts/ci/setup/build-environment.yml`` Ansible playbook will
+set up machines with the environment needed to perform builds and run
+QEMU tests. This playbook consists on the installation of various
+required packages (and a general package update while at it). It
+currently covers a number of different Linux distributions, but it can
+be expanded to cover other systems.
+
+The minimum required version of Ansible successfully tested in this
+playbook is 2.8.0 (a version check is embedded within the playbook
+itself). To run the playbook, execute::
+
+ cd scripts/ci/setup
+ ansible-playbook -i inventory build-environment.yml
+
+Please note that most of the tasks in the playbook require superuser
+privileges, such as those from the ``root`` account or those obtained
+by ``sudo``. If necessary, please refer to ``ansible-playbook``
+options such as ``--become``, ``--become-method``, ``--become-user``
+and ``--ask-become-pass``.
diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
new file mode 100644
index 0000000000..ee088604d1
--- /dev/null
+++ b/scripts/ci/setup/.gitignore
@@ -0,0 +1,2 @@
+inventory
+
diff --git a/scripts/ci/setup/build-environment.yml
b/scripts/ci/setup/build-environment.yml
new file mode 100644
index 0000000000..581c1c75d1
--- /dev/null
+++ b/scripts/ci/setup/build-environment.yml
@@ -0,0 +1,116 @@
+# Copyright (c) 2021 Red Hat, Inc.
+#
+# Author:
+# Cleber Rosa <crosa@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+#
+# This is an ansible playbook file. Run it to set up systems with the
+# environment needed to build QEMU.
+---
+- name: Installation of basic packages to build QEMU
+ hosts: all
+ tasks:
+ - name: Check for suitable ansible version
+ delegate_to: localhost
+ assert:
+ that:
+ - '((ansible_version.major == 2) and (ansible_version.minor >= 8))
or (ansible_version.major >= 3)'
+ msg: "Unsuitable ansible version, please use version 2.8.0 or later"
+
+ - name: Update apt cache / upgrade packages via apt
+ apt:
+ update_cache: yes
+ upgrade: yes
+ when:
+ - ansible_facts['distribution'] == 'Ubuntu'
+
+ - name: Install basic packages to build QEMU on Ubuntu 18.04/20.04
+ package:
+ name:
+ # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+ - ccache
+ - gcc
+ - gettext
+ - git
+ - glusterfs-common
+ - libaio-dev
+ - libattr1-dev
+ - libbrlapi-dev
+ - libbz2-dev
+ - libcacard-dev
+ - libcap-ng-dev
+ - libcurl4-gnutls-dev
+ - libdrm-dev
+ - libepoxy-dev
+ - libfdt-dev
+ - libgbm-dev
+ - libgtk-3-dev
+ - libibverbs-dev
+ - libiscsi-dev
+ - libjemalloc-dev
+ - libjpeg-turbo8-dev
+ - liblzo2-dev
+ - libncurses5-dev
+ - libncursesw5-dev
+ - libnfs-dev
+ - libnss3-dev
+ - libnuma-dev
+ - libpixman-1-dev
+ - librados-dev
+ - librbd-dev
+ - librdmacm-dev
+ - libsasl2-dev
+ - libsdl2-dev
+ - libseccomp-dev
+ - libsnappy-dev
+ - libspice-protocol-dev
+ - libssh-dev
+ - libusb-1.0-0-dev
+ - libusbredirhost-dev
+ - libvdeplug-dev
+ - libvte-2.91-dev
+ - libzstd-dev
+ - make
+ - python3-yaml
+ - python3-sphinx
+ - python3-sphinx-rtd-theme
+ - ninja-build
+ - sparse
+ - xfslibs-dev
+ state: present
+ when:
+ - ansible_facts['distribution'] == 'Ubuntu'
+
+ - name: Install packages to build QEMU on Ubuntu 18.04/20.04 on non-s390x
+ package:
+ name:
+ - libspice-server-dev
+ - libxen-dev
+ state: present
+ when:
+ - ansible_facts['distribution'] == 'Ubuntu'
+ - ansible_facts['architecture'] != 's390x'
+
+ - name: Install basic packages to build QEMU on Ubuntu 18.04
+ package:
+ name:
+ # Originally from tests/docker/dockerfiles/ubuntu1804.docker
+ - clang
+ when:
+ - ansible_facts['distribution'] == 'Ubuntu'
+ - ansible_facts['distribution_version'] == '18.04'
+
+ - name: Install basic packages to build QEMU on Ubuntu 20.04
+ package:
+ name:
+ # Originally from tests/docker/dockerfiles/ubuntu2004.docker
+ - clang-10
+ - genisoimage
+ - liblttng-ust-dev
+ - libslirp-dev
+ - netcat-openbsd
+ when:
+ - ansible_facts['distribution'] == 'Ubuntu'
+ - ansible_facts['distribution_version'] == '20.04'
diff --git a/scripts/ci/setup/inventory.template
b/scripts/ci/setup/inventory.template
new file mode 100644
index 0000000000..2fbb50c4a8
--- /dev/null
+++ b/scripts/ci/setup/inventory.template
@@ -0,0 +1 @@
+localhost
--
2.20.1
- [PULL v5 00/44] testing, build and plugin updates, Alex Bennée, 2021/07/14
- [PULL v5 01/44] Jobs based on custom runners: documentation and configuration placeholder, Alex Bennée, 2021/07/14
- [PULL v5 03/44] Jobs based on custom runners: docs and gitlab-runner setup playbook, Alex Bennée, 2021/07/14
- [PULL v5 05/44] tests/tcg: also disable the signals test for plugins, Alex Bennée, 2021/07/14
- [PULL v5 08/44] build: validate that system capstone works before using it, Alex Bennée, 2021/07/14
- [PULL v5 04/44] Jobs based on custom runners: add job definitions for QEMU's machines, Alex Bennée, 2021/07/14
- [PULL v5 07/44] disable modular TCG on Darwin, Alex Bennée, 2021/07/14
- [PULL v5 02/44] Jobs based on custom runners: build environment docs and playbook,
Alex Bennée <=
- [PULL v5 09/44] gitlab: support for FreeBSD 12, 13 and macOS 11 via cirrus-run, Alex Bennée, 2021/07/14
- [PULL v5 06/44] meson: fix condition for io_uring stubs, Alex Bennée, 2021/07/14
- [PULL v5 10/44] cirrus: delete FreeBSD and macOS jobs, Alex Bennée, 2021/07/14
- [PULL v5 11/44] hw/usb/ccid: remove references to NSS, Alex Bennée, 2021/07/14
- [PULL v5 14/44] tests/docker: use explicit docker.io registry, Alex Bennée, 2021/07/14
- [PULL v5 12/44] tests/docker: don't use BUILDKIT in GitLab either, Alex Bennée, 2021/07/14
- [PULL v5 39/44] docs/devel: tcg-plugins: add execlog plugin description, Alex Bennée, 2021/07/14
- [PULL v5 23/44] tests/docker: expand ubuntu1804 package list, Alex Bennée, 2021/07/14
- [PULL v5 36/44] tcg/plugins: enable by default for most TCG builds, Alex Bennée, 2021/07/14
- [PULL v5 44/44] MAINTAINERS: Added myself as a reviewer for TCG Plugins, Alex Bennée, 2021/07/14