[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[RFC PATCH v2 0/7] tests: run python tests under a venv
From: |
John Snow |
Subject: |
[RFC PATCH v2 0/7] tests: run python tests under a venv |
Date: |
Fri, 10 Jun 2022 18:25:58 -0400 |
Hi, here's another RFC for bringing external Python dependencies to the
QEMU test suite.
This patchset is not without some problems that need to be solved, but
I've been sitting on these long enough and they need to see the light of
day.
Problems I am aware of, and there's a few:
- Ubuntu 18.04 ships with a version of pip that is too old to support
setup.cfg-based installations. We are allowed to drop support for
18.04 by now, but we need a suitable 32bit debian VM configuration to
replace it.
- Multiple VM tests are still failing for me; but they fail with or
without my patches as far as I can tell. I'm having problems with
Haiku and CentOS, primarily -- which I think fail even without my
patches. I'll have more info after the weekend, these tests are SLOW.
- This version of the patch series does not itself enforce any
offline-only behavior for venv creation, but downstreams can modify
any call to 'mkvenv' to pass '--offline'. A more flexible approach
might be to allow an environment variable to be passed that toggles
the switch on.
- iotests will now actually never run mypy or pylint tests by default
anymore, because the bootstrapper won't select those packages by
default, and the virtual environment won't utilize the system packages
-- so iotest 297 will just "skip" all of the time now.
The reason we don't want to install these packages by default is
because we don't want to add dependencies on mypy and pylint for
downstream builds.
With these patches, 297 would still work if you manually opened up the
testing venv and installed suitable mypy/pylint packages there. I
could also add a new optional dependency group, and one could
theoretically invoke a once-per-build-dir command of 'make
check-venv-pylint' to help make the process only semi-manual, but it's
still annoying.
Ideally, the python checks in qemu.git/python/ can handle the same
tests as 297 does -- but we need to give a shorthand invocation like
"make check-python" that is excluded from the default "make check" to
allow block developers to quickly opt-in to the same tests.
I've covered some of the problems here on-list before:
https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg03661.html
...But I haven't quite solved them yet.
That's all for now.
Paolo, can we chat about build system integration next? I want to know
how you envision the integration at this point -- adding different
test-invocation styles (online, offline, etc) may help solve the iotest
297 problem and the iotest self-bootstrap problem.
--js
John Snow (7):
tests: create optional tests/venv dependency groups
tests: pythonize test venv creation
tests: Remove spurious pip warnings on Ubuntu20.04
tests/vm: add venv pre-requisites to VM building recipes
tests: add 'check-venv' as a dependency of 'make check'
iotests: use tests/venv for running tests
iotests: self-bootstrap testing venv
tests/Makefile.include | 32 +++---
tests/mkvenv.py | 187 ++++++++++++++++++++++++++++++++++
tests/qemu-iotests/testenv.py | 25 +++--
tests/requirements.txt | 6 --
tests/setup.cfg | 20 ++++
tests/setup.py | 16 +++
tests/vm/netbsd | 1 +
tests/vm/ubuntu.i386 | 9 +-
8 files changed, 268 insertions(+), 28 deletions(-)
create mode 100644 tests/mkvenv.py
delete mode 100644 tests/requirements.txt
create mode 100644 tests/setup.cfg
create mode 100644 tests/setup.py
--
2.34.3
- [RFC PATCH v2 0/7] tests: run python tests under a venv,
John Snow <=
- [RFC PATCH v2 4/7] tests/vm: add venv pre-requisites to VM building recipes, John Snow, 2022/06/10
- [RFC PATCH v2 3/7] tests: Remove spurious pip warnings on Ubuntu20.04, John Snow, 2022/06/10
- [RFC PATCH v2 2/7] tests: pythonize test venv creation, John Snow, 2022/06/10
- [RFC PATCH v2 1/7] tests: create optional tests/venv dependency groups, John Snow, 2022/06/10
- [RFC PATCH v2 7/7] iotests: self-bootstrap testing venv, John Snow, 2022/06/10
- [RFC PATCH v2 6/7] iotests: use tests/venv for running tests, John Snow, 2022/06/10
- [RFC PATCH v2 5/7] tests: add 'check-venv' as a dependency of 'make check', John Snow, 2022/06/10