Re: [PATCH v7 31/31] gitlab: add python linters to CI

[Vladimir Sementsov-Ogievskiy]

26.05.2021 21:24, John Snow wrote:
> On 5/26/21 5:33 AM, Vladimir Sementsov-Ogievskiy wrote:
> > 26.05.2021 03:24, John Snow wrote:
> > > Add a python container that contains just enough juice for us to run the python
> > > code quality analysis tools.
> > >
> > > Base this container on fedora, because fedora has very convenient
> > > packaging for testing multiple python versions.
> > >
> > > Add two tests:
> > >
> > > check-python-pipenv uses pipenv to test a frozen, very explicit set of
> > > packages against our minimum supported python version, Python 3.6. This
> > > test is not allowed to fail.
> > >
> > > check-python-tox uses tox to install the latest versions of required
> > > python dependencies against a wide array of Python versions from 3.6 to
> > > 3.9, even including the yet-to-be-released Python 3.10. This test is
> > > allowed to fail with a warning.
> > >
> > > Signed-off-by: John Snow <jsnow@redhat.com>
> > > ---
> > >   .gitlab-ci.d/containers.yml            |  5 +++++
> > >   .gitlab-ci.yml                         | 26 ++++++++++++++++++++++++++
> > >   tests/docker/dockerfiles/python.docker | 18 ++++++++++++++++++
> > >   3 files changed, 49 insertions(+)
> > >   create mode 100644 tests/docker/dockerfiles/python.docker
> > >
> > > diff --git a/.gitlab-ci.d/containers.yml b/.gitlab-ci.d/containers.yml
> > > index 765408ae274..05ebd4dc11d 100644
> > > --- a/.gitlab-ci.d/containers.yml
> > > +++ b/.gitlab-ci.d/containers.yml
> > > @@ -242,3 +242,8 @@ amd64-opensuse-leap-container:
> > >     extends: .container_job_template
> > >     variables:
> > >       NAME: opensuse-leap
> > > +
> > > +python-container:
> > > +  extends: .container_job_template
> > > +  variables:
> > > +    NAME: python
> > > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > > index f718b61fa78..cc2a3935c62 100644
> > > --- a/.gitlab-ci.yml
> > > +++ b/.gitlab-ci.yml
> > > @@ -789,6 +789,32 @@ check-patch:
> > >       GIT_DEPTH: 1000
> > >     allow_failure: true
> > > +
> > > +check-python-pipenv:
> > > +  stage: test
> > > +  image: $CI_REGISTRY_IMAGE/qemu/python:latest
> > > +  script:
> > > +    - cd python
> > > +    - make venv-check
> > > +  variables:
> > > +    GIT_DEPTH: 1000
> >
> > Hmm, interesting, why we need depth = 1000? gitlab recommends to keep that number 
> > "small like 10" https://docs.gitlab.com/ee/ci/large_repositories/
>
> Yeah, I don't. Just copy-pasted and didn't consider it. I can set it to "1". 
> The default is apparently 50 and I don't need that either.
>
> > > +  needs:
> > > +    job: python-container
> > > +
> > > +
> > > +check-python-tox:
> > > +  stage: test
> > > +  image: $CI_REGISTRY_IMAGE/qemu/python:latest
> > > +  script:
> > > +    - cd python
> > > +    - make check-tox
> > > +  variables:
> > > +    GIT_DEPTH: 1000
> > > +  needs:
> > > +    job: python-container
> > > +  allow_failure: true
> > > +
> > > +
> > >   check-dco:
> > >     stage: build
> > >     image: $CI_REGISTRY_IMAGE/qemu/centos8:latest
> > > diff --git a/tests/docker/dockerfiles/python.docker 
> > > b/tests/docker/dockerfiles/python.docker
> > > new file mode 100644
> > > index 00000000000..56d88417df4
> > > --- /dev/null
> > > +++ b/tests/docker/dockerfiles/python.docker
> > > @@ -0,0 +1,18 @@
> > > +# Python library testing environment
> > > +
> > > +FROM fedora:latest
> > > +MAINTAINER John Snow <jsnow@redhat.com>
> > > +
> > > +# Please keep this list sorted alphabetically
> > > +ENV PACKAGES \
> > > +    gcc \
> >
> > hmm, interesting, why you need gcc to run python linters?
>
> build requisite for PyPI packages in the event that PyPI only has a sdist and 
> not a bdist for a given dependency during installation.

i.e. some packages are compiled during installation?

> Found that out the hard way.

Worth leaving the comment somewhere? (not worth any kind of resending of course)

> > > +    make \
> > > +    pipenv \
> > > +    python3 \
> > > +    python3-pip \
> > > +    python3-tox \
> > > +    python3-virtualenv \
> > > +    python3.10
> > > +
> > > +RUN dnf install -y $PACKAGES
> > > +RUN rpm -q $PACKAGES | sort > /packages.txt
> >
> >
> > weak, as I'm far from understanding the details, I can only check that it looks 
> > similar with nearby files and entities:
> >
> > Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> Thanks!


--
Best regards,
Vladimir