Re: [PATCH 0/7] python: create installable package

[John Snow]

On 6/19/20 12:44 PM, Kevin Wolf wrote:
> Am 19.06.2020 um 17:04 hat John Snow geschrieben:
>> On 6/18/20 5:23 AM, Kevin Wolf wrote:
>>> Am 17.06.2020 um 22:27 hat John Snow geschrieben:
>>>>> In the Avocado project, we have a `make develop` rule that does that
>>>>> for the main setup.py file, and for all plugins we carry on the same
>>>>> tree, which is similar in some regards to the "not at the project root
>>>>> directory" situation here with "qemu/python/setup.py".
>>>>>
>>>>
>>>> Ah, yeah. If we're going this far, I'd prefer using a VENV over
>>>> modifying the user's environment. That way you can blast it all away
>>>> with a `make distclean`.
>>>>
>>>> Maybe the "make develop" target could even use the presence of a .venv
>>>> directory to know when it needs to make the environment or not ...
>>> [..]
>>>> For QEMU developers, installing with develop is going to be the smart
>>>> way to go. When your git tree is updated, your package will be updated
>>>> along with it. You can do it once and then probably forget about it.
>>>
>>> I don't think we can make this a manual step at all. Building QEMU
>>> requires running some Python scripts (e.g. the QAPI generator), so the
>>> setup needs to be done either in configure or in a Makefile target that
>>> is specified as a dependency of any rule that would run a Python script.
>>> Building QEMU once would then be enough.
>>
>> I am imagining that we might treat "building" and "testing" separately
>> -- as it is, builds require python3.5 and tests requires 3.6 which
>> definitely necessitates two distinct environments.
> 
> I'm not sure what the exact definition of "end of life" of a distro is
> that we're using. I seem to remember that the reason for using Python
> 3.5 was Debian Stretch. Its official end of life is in about three
> weeks, but then there is still some LTS thing with reduced support done
> by a different group.
> 
> If we read our policy literally and use the regular end of life, I guess
> we could just move QEMU to 3.6 for everything.
> 

I think we have kinda-sorta-agreed to exclude the third-party LTS
support. I think we will be able to move to Python 3.6 shortly.

That'd solve one problem.

>> I will admit that I haven't constructed a full, coherent vision of
>> python management that encapsulates both building ad testing yet. For
>> example, should configure/make expect to be run inside of a venv, or
>> should they expect to create and then enter the venv? That's not clear
>> to me yet. I'm simultaneously trying to work out with Peter Maydell how
>> the sphinx dependency should work. Sphinx is presently our only python
>> dependency for our build environment.)
> 
> It's kind of obvious that this can't require user interaction because we
> want ./configure; make to work. So I guess this means the venv needs to
> be set up automatically by configure/make?
> 
>> Perhaps starting with the testing step is a good starting point and we
>> can use an implicit dependency on a `make develop` style step so it
>> happens automatically.
>>
>> (But perhaps keeping it as a standalone target that CAN be invoked
>> manually would be nice if you want to do some more intensive debugging
>> or development of new tests.)
> 
> Yes. And you'll have many dependencies on it, so it would be a separate
> target anyway.
> 
>>> Doing it automatically also means that we have to keep things local to
>>> the QEMU directory rather than installing them globally into the user
>>> directory. This is desirable anyway: Most of us deal with more than one
>>> QEMU source tree, so conflicts would be inevitable.
>>
>> I think it should be easy enough to put the VENV in the build directory
>> to prevent cross-contamination.
> 
> Sure. I'm not overly familiar with all of this, but I guess my point was
> just that a venv is needed rather than a global installation into the
> user directory. If nobody ever suggested the latter, blame the
> misunderstanding on my non-existent experience with more complex Python
> setups.
> 

Python doesn't make it easy to understand, I think.

I'll head along in this direction: We want testing to use a venv that
exists in the build directory and we want to automate its creation and
usage.

I am still working out the role of Python/VENVs at configure time with
Peter Maydell in another thread which may answer the other half of the
equation for me.

--js

> Kevin
>