Re: [PATCH 2/7] python/qemu: formalize as package

[John Snow]

On 6/5/20 10:40 AM, Vladimir Sementsov-Ogievskiy wrote:
> 03.06.2020 03:15, John Snow wrote:
>> NB: I am choosing Python 3.6 here. Although our minimum requirement is
>> 3.5, this code is used only by iotests (so far) under which we have been
>> using a minimum version of 3.6.
>>
>> 3.6 is being preferred here for variable type hint capability, which
>> enables us to use mypy for this package.
>>
>> RFC: This uses the version tags of the parent tree here, so packages
>> will be installed as e.g. 5.0.0, 5.1.0-rc0, etc.
>>
>> Pros:
>>   - Easy to tell which versions of QEMU it supports
>>   - Simple
>>
>> Cons:
>>   - Implies semver, which we do NOT follow for QEMU releases
>>   - Implies the package is in a stable state
> 
> Necessarily? Couldn't we state Development Status: Alpha, even with
> version 5.1.0 ?
> 

It's a good question, actually...

I had thought that the Python packaging ecosystem implied semver, but I
don't see proof of that right now, actually.

PEP440 https://www.python.org/dev/peps/pep-0440/#semantic-versioning
says only that it considers many of the same cases as python versioning
and that it's more prescriptive.

However, in the "Version scheme" section, PEP440 says:

"All numeric components MAY be zero. Except as described below for the
release segment, a numeric component of zero has no special significance
aside from always being the lowest possible value in the version ordering."

I actually don't know what special significance a release segment of 0
has, though -- I don't see that later in the doc.

Semver is described here: https://semver.org/

"Major version zero (0.y.z) is for initial development. Anything MAY
change at any time. The public API SHOULD NOT be considered stable."

I guess we're not beholden to it. Still, it seems common in the Python
world to gate based on major version. It may present packaging
difficulties. I'll try to research more. Maybe I'll reach out to the
PyPI folks and see if I can get advice here.

>>
>>
>> We could also start a separate versioning for just the Python SDK at
>> e.g. 0.1;
>>
>> Pros:
>>   - We can use semver, which is expected of Python packaging
>>   - Allows us to break compatibility for 0.x releases
>>
>> Cons:
>>   - More complex, the mapping from SDK version to QEMU version
>>     is less obvious
>>   - Requires someone to manage a secondary version commit for
>>     the Python SDK.
>>
>> Or, perhaps, we could start versioning with 0.5.0.0, 0.5.1.0, etc to
>> combine a bit of both flavors; bumping the major version number only
>> when incompatible changes to the Python interface itself are made,
>> treating the major version number more like an epoch.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>>   python/README.rst |  6 ++++++
>>   python/setup.py   | 50 +++++++++++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 56 insertions(+)
>>   create mode 100644 python/README.rst
>>   create mode 100755 python/setup.py
>>
>> diff --git a/python/README.rst b/python/README.rst
>> new file mode 100644
>> index 00000000000..25f6d93fd5f
>> --- /dev/null
>> +++ b/python/README.rst
>> @@ -0,0 +1,6 @@
>> +QEMU Python Tooling
>> +-------------------
>> +
>> +This package provides QEMU tooling used by the QEMU project to build,
>> +configure, and test QEMU. It is not a fully-fledged SDK and it is
>> subject
>> +to change at any time.
>> diff --git a/python/setup.py b/python/setup.py
>> new file mode 100755
>> index 00000000000..f897ceac970
>> --- /dev/null
>> +++ b/python/setup.py
>> @@ -0,0 +1,50 @@
>> +#!/usr/bin/env3 python
> 
> env python3 you mean
> 

Oh wow. The funny thing is that it works anyway when you use pip!

> with it fixed:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
> 
>> +"""
>> +QEMU tooling installer script
>> +Copyright (c) 2020 John Snow for Red Hat, Inc.
>> +"""
>> +
>> +import setuptools
>> +
>> +def main():
>> +    """
>> +    QEMU tooling installer
>> +    """
>> +
>> +    kwargs = {
>> +        'name': 'qemu',
>> +        'use_scm_version': {
>> +            'root': '..',
>> +            'relative_to': __file__,
>> +        },
>> +        'maintainer': 'QEMU Developer Team',
>> +        'maintainer_email': 'qemu-devel@nongnu.org',
>> +        'url': 'https://www.qemu.org/',
>> +        'download_url': 'https://www.qemu.org/download/',
>> +        'packages': setuptools.find_namespace_packages(),
>> +        'description': 'QEMU Python Build, Debug and SDK tooling.',
>> +        'classifiers': [
>> +            'Development Status :: 5 - Production/Stable',
> 
> Could we use "3 - Alpha" ?
> 

Yes. I used 5 for the placeholder package to explain QEMU's stability,
but I should describe the SDK stability instead.

>> +            'License :: OSI Approved :: GNU General Public License v2
>> (GPLv2)',
>> +            'Natural Language :: English',
>> +            'Operating System :: OS Independent',
>> +        ],
>> +        'platforms': [],
>> +        'keywords': [],
>> +        'setup_requires': [
>> +            'setuptools',
>> +            'setuptools_scm',
>> +        ],
> 
> Hmm, documentation says:
> 
>    Warning Using setup_requires is discouraged in favor of PEP-518
> 
> did you consider this thing?
> 

Oh, the TOML thing ... OK, I need to research this more. I tried using
it a few times and I couldn't observe it to be actually working.

It doesn't seem to be in common, widespread usage in the ecosystem yet.

(Also, and I'm serious: it requires a TOML module that isn't built into
Python, so it has some detriments, I think.)

I'll look into this more ardently.

>> +        'install_requires': [
>> +        ],
>> +        'python_requires': '>=3.6',
>> +        'long_description_content_type': 'text/x-rst',
>> +    }
>> +
>> +    with open("README.rst", "r") as fh:
> 
> You use '' for all other strings (except for doc-strings) in this file.
> Maybe use '' here too?
> 

Yes, I should be consistent.

This is not based on any PEP, but I tend to prefer using single quotes
for static strings, and double quotes for format strings -- except when
convenient to deviate for reasons of avoiding backslashes.

>> +        kwargs['long_description'] = fh.read()
>> +
>> +    setuptools.setup(**kwargs)
>> +
>> +if __name__ == '__main__':
>> +    main()
>>
> 
> Hmm in examples in documentations I always see something like this:
> 
>> from setuptools import setup, find_namespace_packages
>>
>> setup(
>>     name='mynamespace-subpackage-a',
>>     ...
>>     packages=find_namespace_packages(include=['mynamespace.*'])
>> )
> 
> How much is it better (or popular) to use __name__ == __main__ style for
> setup.py?
> We are not going to use it as module to import main somewhere...
> 

Mostly just habit. It's not important here, but if you should so happen
to check your setup.py? with pylint, using the __main__ pattern keeps the
check passing.