Re: [PATCH 02/10] python: drop pipenv

[John Snow]

On Thu, Mar 16, 2023 at 4:54 AM Philippe Mathieu-Daudé
<philmd@linaro.org> wrote:
>
> On 16/3/23 00:02, John Snow wrote:
> > On Wed, Mar 15, 2023 at 5:17 PM Philippe Mathieu-Daudé
> > <philmd@linaro.org> wrote:
> >>
> >> +Jan
> >>
> >> On 22/2/23 15:37, Paolo Bonzini wrote:
> >>> From: John Snow <jsnow@redhat.com>
> >>>
> >>> The pipenv tool was nice in theory, but in practice it's just too hard
> >>> to update selectively, and it makes using it a pain. The qemu.qmp repo
> >>> dropped pipenv support a while back and it's been functioning just fine,
> >>> so I'm backporting that change here to qemu.git.
> >>>
> >>> Signed-off-by: John Snow <jsnow@redhat.com>
> >>> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> >>> ---
> >>>    .gitlab-ci.d/static_checks.yml         |   4 +-
> >>>    python/.gitignore                      |   4 +-
> >>>    python/Makefile                        |  53 ++--
> >>>    python/Pipfile                         |  13 -
> >>>    python/Pipfile.lock                    | 347 -------------------------
> >>>    python/README.rst                      |   3 -
> >>>    python/setup.cfg                       |   4 +-
> >>>    python/tests/minreqs.txt               |  45 ++++
> >>>    tests/docker/dockerfiles/python.docker |   1 -
> >>>    9 files changed, 86 insertions(+), 388 deletions(-)
> >>>    delete mode 100644 python/Pipfile
> >>>    delete mode 100644 python/Pipfile.lock
> >>>    create mode 100644 python/tests/minreqs.txt
>
>
> >>> diff --git a/python/setup.cfg b/python/setup.cfg
> >>> index 564181570654..9e923d97628f 100644
> >>> --- a/python/setup.cfg
> >>> +++ b/python/setup.cfg
> >>> @@ -33,9 +33,7 @@ packages =
> >>>    * = py.typed
> >>>
> >>>    [options.extras_require]
> >>> -# For the devel group, When adding new dependencies or bumping the 
> >>> minimum
> >>> -# version, use e.g. "pipenv install --dev pylint==3.0.0".
> >>> -# Subsequently, edit 'Pipfile' to remove e.g. 'pylint = "==3.0.0'.
> >>> +# Remember to update tests/minreqs.txt if changing anything below:
> >>>    devel =
> >>>        avocado-framework >= 90.0
> >>
> >> Here >= 90,
> >
> > Yes. The "devel" group for the python packages here requires
> > avocado-framework 90.0 or better... to run the tests *on the python
> > package*.
> >
> >>
> >>>        flake8 >= 3.6.0
> >>> diff --git a/python/tests/minreqs.txt b/python/tests/minreqs.txt
> >>> new file mode 100644
> >>> index 000000000000..dfb8abb155f4
> >>> --- /dev/null
> >>> +++ b/python/tests/minreqs.txt
> >>> @@ -0,0 +1,45 @@
> >>> +# This file lists the ***oldest possible dependencies*** needed to run
> >>> +# "make check" successfully under ***Python 3.6***. It is used primarily
> >>> +# by GitLab CI to ensure that our stated minimum versions in setup.cfg
> >>> +# are truthful and regularly validated.
> >>> +#
> >>> +# This file should not contain any dependencies that are not expressed
> >>> +# by the [devel] section of setup.cfg, except for transitive
> >>> +# dependencies which must be enumerated here explicitly to eliminate
> >>> +# dependency resolution ambiguity.
> >>> +#
> >>> +# When adding new dependencies, pin the very oldest non-yanked version
> >>> +# on PyPI that allows the test suite to pass.
> >>> +
> >>> +# Dependencies for the TUI addon (Required for successful linting)
> >>> +urwid==2.1.2
> >>> +urwid-readline==0.13
> >>> +Pygments==2.9.0
> >>> +
> >>> +# Dependencies for FUSE support for qom-fuse
> >>> +fusepy==2.0.4
> >>> +
> >>> +# Test-runners, utilities, etc.
> >>> +avocado-framework==90.0
> >>
> >> ... and here == 90.
> >
> > Yes. This is the minimum requirements file for the purposes of testing
> > the python code via CI ("python-check-minreqs").
> > It installs *exactly* version 90.0 to ensure that there are no
> > avocado-framework features used in later versions that have
> > accidentally slipped in.
> >
> >>
> >> Anyhow I'm surprised by unreviewed commit 4320f7172f
> >> ("python: bump avocado to v90.0") and we still have:
> >
> > The implication being that it wouldn't have passed review?
>
> I want to use Avocado 90+ since more than 1 year now as this
> would make my maintainer life easier, but "something" was missing
> and Cleber said he'd address that, then we could move to v90.
>
> I suppose the qemu.qmp package is not tested on Darwin/macOS,
> or is not tested with Avocado.
>

Ah, that's true. I don't have a mac and the CI tests *on* the qemu.qmp
code run under Fedora. I don't know what stops us from running them on
mac at all, but I haven't tried.

> >> tests/requirements.txt:5:avocado-framework==88.1
> >
> > Developing the python code needs avocado >= 90,
>
> What do you mean by "Developing the python code"?
>

cd qemu.git/python && make check-dev

The [devel] group of requisites are used to install requisites for
linting and running various tools on the python code itself. It's also
used for "check-tox" or "check-minreqs" which all behave slightly
differently, but ultimately do the same thing. Try "make help" to see
a nice summary.

> > running the avocado tests needs avocado==88.1.
> >
> >>
> >> 1/ How do you run Avocado tests out of tests/ ?
> >
> > "make check-avocado", the same as anyone else, I assume.
> >
> >>
> >> 2/ Can we use Avocado on Darwin/macOS now? I thought we
> >> needed one series from Cleber [*] for that, which is why
> >> QEMU is the last project using the 'old' runner (as opposed
> >> to the 'new' runner which is the upstream /current/ one).
> >
> > I have no idea. I'm going to guess that you're wondering if you can
> > run the avocado *qemu* tests on Darwin/macOS, and I wouldn't know - I
> > don't develop avocado or those tests, and I don't have a mac.
>
> Well currently we can not run Avocado on Darwin/macOS (at least since
> macOS Big Sur which is where I started).
>