[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] tests/acceptance: allow control over tags during check-accep
From: |
Philippe Mathieu-Daudé |
Subject: |
Re: [PATCH] tests/acceptance: allow control over tags during check-acceptance |
Date: |
Tue, 6 Jul 2021 22:46:10 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.11.0 |
On 7/6/21 9:06 PM, Willian Rampazzo wrote:
> Although it is possible to run a specific test using the avocado
> command-line, a user may want to use a specific tag while running the
> ``make check-acceptance`` during the development or debugging.
>
> This allows using the TAGS environment variable where the user takes
> total control of which tests should run based on the tags defined.
>
> This also makes the check-acceptance command flexible to restrict tests
> based on tags while running on CI.
>
> e.g.:
>
> TAGS="foo bar baz" make check-acceptance
I'm worried 'TAGS' is a bit too generic...
Maybe rename AVOCADO_TAGS -> AVOCADO_CMDLINE_TAGS and
TAGS -> AVOCADO_TAGS?
> Signed-off-by: Willian Rampazzo <willianr@redhat.com>
> ---
> docs/devel/testing.rst | 7 +++++++
> tests/Makefile.include | 10 ++++++++--
> 2 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 4e42392810..6e03c3449b 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -760,6 +760,13 @@ in the current directory, tagged as "quick", run:
>
> avocado run -t quick .
>
> +It is also possible to run tests based on tags using the
> +``make check-acceptance`` command and the ``TAGS`` environment variable:
> +
> +.. code::
> +
> + TAGS=quick make check-acceptance
> +
> The ``avocado_qemu.Test`` base test class
> -----------------------------------------
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 8f220e15d1..5869ab8a04 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -92,7 +92,11 @@ TESTS_RESULTS_DIR=$(BUILD_DIR)/tests/results
> # Any number of command separated loggers are accepted. For more
> # information please refer to "avocado --help".
> AVOCADO_SHOW=app
> -AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter %-softmmu,$(TARGETS)))
> +ifndef TAGS
> + AVOCADO_TAGS=$(patsubst %-softmmu,-t arch:%, $(filter
> %-softmmu,$(TARGETS)))
> +else
> + AVOCADO_TAGS=$(addprefix -t , $(TAGS))
> +endif
>
> $(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> $(call quiet-command, \
> @@ -125,10 +129,12 @@ get-vm-image-fedora-31-%: check-venv
> get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%,
> $(FEDORA_31_DOWNLOAD))
>
> check-acceptance: check-venv $(TESTS_RESULTS_DIR) get-vm-images
> + @echo "AVOCADO_TAGS $(AVOCADO_TAGS)"
> $(call quiet-command, \
> $(TESTS_VENV_DIR)/bin/python -m avocado \
> --show=$(AVOCADO_SHOW) run
> --job-results-dir=$(TESTS_RESULTS_DIR) \
> - --filter-by-tags-include-empty
> --filter-by-tags-include-empty-key \
> + $(if $(TAGS),, --filter-by-tags-include-empty \
> + --filter-by-tags-include-empty-key) \
> $(AVOCADO_TAGS) \
> $(if $(GITLAB_CI),,--failfast) tests/acceptance, \
> "AVOCADO", "tests/acceptance")
>