[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner
From: |
Cleber Rosa |
Subject: |
Re: [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook |
Date: |
Tue, 29 Jun 2021 20:04:20 -0400 |
On Wed, Jun 9, 2021 at 1:46 PM Willian Rampazzo <wrampazz@redhat.com> wrote:
>
> On Tue, Jun 8, 2021 at 12:14 AM Cleber Rosa <crosa@redhat.com> wrote:
> >
> > To have the jobs dispatched to custom runners, gitlab-runner must
> > be installed, active as a service and properly configured. The
> > variables file and playbook introduced here should help with those
> > steps.
> >
> > The playbook introduced here covers the Linux distributions and
> > has been primarily tested on OS/machines that the QEMU project
> > has available to act as runners, namely:
> >
> > * Ubuntu 20.04 on aarch64
> > * Ubuntu 18.04 on s390x
> >
> > But, it should work on all other Linux distributions. Earlier
> > versions were tested on FreeBSD too, so chances of success are
> > high.
> >
> > Signed-off-by: Cleber Rosa <crosa@redhat.com>
> > ---
> > docs/devel/ci.rst | 57 ++++++++++++++++++++++++++++
> > scripts/ci/setup/.gitignore | 1 +
> > scripts/ci/setup/gitlab-runner.yml | 61 ++++++++++++++++++++++++++++++
> > scripts/ci/setup/vars.yml.template | 12 ++++++
> > 4 files changed, 131 insertions(+)
> > create mode 100644 scripts/ci/setup/.gitignore
> > create mode 100644 scripts/ci/setup/gitlab-runner.yml
> > create mode 100644 scripts/ci/setup/vars.yml.template
> >
> > diff --git a/docs/devel/ci.rst b/docs/devel/ci.rst
> > index 35c6b5e269..bbd89e54d7 100644
> > --- a/docs/devel/ci.rst
> > +++ b/docs/devel/ci.rst
> > @@ -56,3 +56,60 @@ To run the playbook, execute::
> >
> > cd scripts/ci/setup
> > ansible-playbook -i inventory build-environment.yml
> > +
> > +gitlab-runner setup and registration
> > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > +
> > +The gitlab-runner agent needs to be installed on each machine that
> > +will run jobs. The association between a machine and a GitLab project
> > +happens with a registration token. To find the registration token for
> > +your repository/project, navigate on GitLab's web UI to:
> > +
> > + * Settings (the gears like icon), then
>
> * Settings (the gears like icon in the end of the left menu), then
>
ACK. What about:
" * Settings (the gears-like icon at the bottom of the left hand side
vertical toolbar), then"
> I took some time to find it as it was hidden at the end of the scrolling.
>
> > + * CI/CD, then
> > + * Runners, and click on the "Expand" button, then
> > + * Under "Set up a specific Runner manually", look for the value under
> > + "Use the following registration token during setup"
>
> For me, it shows: "And this registration token:"
>
Right, it must have changed. Updating it on v7.
> > +
> > +Copy the ``scripts/ci/setup/vars.yml.template`` file to
> > +``scripts/ci/setup/vars.yml``. Then, set the
> > +``gitlab_runner_registration_token`` variable to the value obtained
> > +earlier.
> > +
> > +.. note:: gitlab-runner is not available from the standard location
> > + for all OS and architectures combinations. For some systems,
> > + a custom build may be necessary. Some builds are avaiable
>
> If you keep this block (see comment from Wainer), s/avaiable/available/
>
I'd rather remove it, but thanks for spotting the typo.
> > + at https://cleber.fedorapeople.org/gitlab-runner/ and this
> > + URI may be used as a value on ``vars.yml``
> > +
> > +To run the playbook, execute::
> > +
> > + cd scripts/ci/setup
> > + ansible-playbook -i inventory gitlab-runner.yml
> > +
> > +Following the registration, it's necessary to configure the runner tags,
> > +and optionally other configurations on the GitLab UI. Navigate to:
> > +
> > + * Settings (the gears like icon), then
> > + * CI/CD, then
> > + * Runners, and click on the "Expand" button, then
> > + * "Runners activated for this project", then
> > + * Click on the "Edit" icon (next to the "Lock" Icon)
> > +
> > +Under tags, add values matching the jobs a runner should run. For a
> > +Ubuntu 20.04 aarch64 system, the tags should be set as::
> > +
> > + ubuntu_20.04,aarch64
> > +
> > +Because the job definition at ``.gitlab-ci.d/custom-runners.yml``
> > +would contain::
> > +
> > + ubuntu-20.04-aarch64-all:
> > + tags:
> > + - ubuntu_20.04
> > + - aarch64
> > +
> > +It's also recommended to:
> > +
> > + * increase the "Maximum job timeout" to something like ``2h``
> > + * give it a better Description
> > diff --git a/scripts/ci/setup/.gitignore b/scripts/ci/setup/.gitignore
> > new file mode 100644
> > index 0000000000..f112d05dd0
> > --- /dev/null
> > +++ b/scripts/ci/setup/.gitignore
> > @@ -0,0 +1 @@
> > +vars.yml
> > \ No newline at end of file
> > diff --git a/scripts/ci/setup/gitlab-runner.yml
> > b/scripts/ci/setup/gitlab-runner.yml
> > new file mode 100644
> > index 0000000000..98dab92bb5
> > --- /dev/null
> > +++ b/scripts/ci/setup/gitlab-runner.yml
> > @@ -0,0 +1,61 @@
> > +---
> > +- name: Installation of gitlab-runner
> > + hosts: all
> > + vars_files:
> > + - vars.yml
> > + tasks:
> > + - debug:
> > + msg: 'Checking for a valid GitLab registration token'
> > + failed_when: "gitlab_runner_registration_token ==
> > 'PLEASE_PROVIDE_A_VALID_TOKEN'"
> > +
> > + - name: Create a group for the gitlab-runner service
> > + group:
> > + name: gitlab-runner
> > +
> > + - name: Create a user for the gitlab-runner service
> > + user:
> > + user: gitlab-runner
> > + group: gitlab-runner
> > + comment: GitLab Runner
> > + home: /home/gitlab-runner
> > + shell: /bin/bash
> > +
> > + - name: Remove the .bash_logout file when on Ubuntu systems
> > + file:
> > + path: /home/gitlab-runner/.bash_logout
> > + state: absent
> > + when: "ansible_facts['distribution'] == 'Ubuntu'"
> > +
> > + - name: Set the Operating System for gitlab-runner
> > + set_fact:
> > + gitlab_runner_os: "{{ ansible_facts[\"system\"]|lower }}"
> > + - debug:
> > + msg: gitlab-runner OS is {{ gitlab_runner_os }}
> > +
> > + - name: Set the architecture for gitlab-runner
> > + set_fact:
> > + gitlab_runner_arch: "{{
> > ansible_to_gitlab_arch[ansible_facts[\"architecture\"]] }}"
> > + - debug:
> > + msg: gitlab-runner arch is {{ gitlab_runner_arch }}
> > +
> > + - name: Download the matching gitlab-runner
> > + get_url:
> > + dest: /usr/local/bin/gitlab-runner
> > + url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{
> > gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{
> > gitlab_runner_arch }}"
> > + owner: gitlab-runner
> > + group: gitlab-runner
> > + mode: u=rwx,g=rwx,o=rx
> > +
> > + - name: Register the gitlab-runner
> > + command: "/usr/local/bin/gitlab-runner register --non-interactive
> > --url {{ gitlab_runner_server_url }} --registration-token {{
> > gitlab_runner_registration_token }} --executor shell --tag-list {{
> > ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower
> > }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{
> > ansible_facts[\"distribution\"] }} {{
> > ansible_facts[\"distribution_version\"] }} {{
> > ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"] }})'"
> > +
> > + - name: Install the gitlab-runner service using its own functionality
> > + command: /usr/local/bin/gitlab-runner install --user gitlab-runner
> > --working-directory /home/gitlab-runner
> > + register: gitlab_runner_install_service_result
> > + failed_when: "gitlab_runner_install_service_result.rc != 0 and
> > \"already exists\" not in gitlab_runner_install_service_result.stderr"
> > +
> > + - name: Enable the gitlab-runner service
> > + service:
> > + name: gitlab-runner
> > + state: started
> > + enabled: yes
> > diff --git a/scripts/ci/setup/vars.yml.template
> > b/scripts/ci/setup/vars.yml.template
> > new file mode 100644
> > index 0000000000..e48089761f
> > --- /dev/null
> > +++ b/scripts/ci/setup/vars.yml.template
> > @@ -0,0 +1,12 @@
> > +# The version of the gitlab-runner to use
> > +gitlab_runner_version: 13.12.0
> > +# The URL of the gitlab server to use, usually https://gitlab.com unless
> > you're
> > +# using a private GitLab instance
> > +gitlab_runner_server_url: https://gitlab.com
> > +# A mapping of the ansible to gitlab architecture nomenclature
> > +ansible_to_gitlab_arch:
> > + x86_64: amd64
> > + aarch64: arm64
> > + s390x: s390x
> > +# A unique token made available by GitLab to your project for registering
> > runners
> > +gitlab_runner_registration_token: PLEASE_PROVIDE_A_VALID_TOKEN
> > --
> > 2.25.4
> >
>
> With my suggestions and Wainer's:
>
> Reviewed-by: Willian Rampazzo <willianr@redhat.com>
> Tested-by: Willian Rampazzo <willianr@redhat.com>
>
- Re: [PATCH v6 2/4] Jobs based on custom runners: build environment docs and playbook, (continued)
- [PATCH v6 1/4] Jobs based on custom runners: documentation and configuration placeholder, Cleber Rosa, 2021/06/07
- [PATCH v6 3/4] Jobs based on custom runners: docs and gitlab-runner setup playbook, Cleber Rosa, 2021/06/07
- [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines, Cleber Rosa, 2021/06/07
- Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines, Stefan Hajnoczi, 2021/06/09
- Re: [PATCH v6 4/4] Jobs based on custom runners: add job definitions for QEMU's machines, Cleber Rosa, 2021/06/29