Re: [PATCH v2 11/15] qemu-common: move scripts/qapi

[Markus Armbruster]

Marc-André Lureau <marcandre.lureau@gmail.com> writes:

> Hi
>
> On Thu, Aug 11, 2022 at 10:52 AM Markus Armbruster <armbru@redhat.com>
> wrote:
>
>> Marc-André Lureau <marcandre.lureau@gmail.com> writes:
>>
>> > Hi
>> >
>> > On Fri, Aug 5, 2022 at 12:12 PM Markus Armbruster <armbru@redhat.com>
>> wrote:
>> >
>> >> marcandre.lureau@redhat.com writes:
>> >>
>> >> > From: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >> >
>> >> > This is just moving qapi-gen.py and related subdir to qemu-common, to
>> >> > ease review and proceed step by step. The following patches will move
>> >> > related necessary code, tests etc.
>> >> >
>> >> > Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com>
>> >>
>> >> As moved files tend to become low-level annoyances for a long time, I'd
>> >> like to understand why you want to move them.  The commit message says
>> >> "to ease review", which I suspect isn't the real reason.  Perhaps you
>> >> explained all that elsewhere already, but I missed it.
>> >>
>> >>
>> >>
>> > The end goal is to split some projects, such as qemu-ga, to standalone
>> > meson projects/subprojects. We will be able to build them independently
>> > from the rest of QEMU, and later on perhaps handle them outside of QEMU
>> > main repository. To achieve this, I first introduce a qemu-common
>> > subproject, where qapi and common units are provided. You can check
>> > https://gitlab.com/marcandre.lureau/qemu/-/commits/qga for a sneak peek at
>> > current end result.
>>
>> I worry this move of the QAPI generator code into
>> subjprojects/common/scripts/qapi/ will be followed by a move into its
>> own subproject.
>>
>
> Do you mean: it could be moved again to another smaller subproject? not
> really, see below
>
>
>> Ignorant question: could we turn the QAPI generator into a subproject in
>> place?
>>
>
> If it's just the generator, probably the target would then be a python
> project (not meson), similar to python-qemu-qmp.
>
> But I don't see much point, since it's not really a standalone python
> module, it generates code, and that code needs most of what is in
> qemu-common (see
> https://gitlab.com/marcandre.lureau/qemu/-/tree/qga/subprojects/qemu-common).
> It's best to have it together imho. Maybe we can consider a different
> naming or to be more careful not to add stuff that is not strictly needed
> by qapi?

I had a look at subjprojects/qemu-common in your qga branch.  Contents:

* Subproject machinery

* Some common headers (glib-compat.h), but not others (qemu/osdep.h).  I
  guess it's whatever subjproject code needs.

  Is subprojects/qemu-common/include/block/nvme.h there by accident?

* Most of the QObject subsystem

  qobject/block-qdict.c is left behind.

* Most of the QAPI subsystem

  Some visitors left behind: opts, forward, string input / output.  Hmm,
  only the .c, the .h are in the subjproject.  Accident?

  A bit of HMP support left behind.

* Parts of util/ and include/qemu/

  Error reporting, key-value CLI, some C utilities, but not others
  (e.g. qemu/atomic.h, but not qemu/atomic128.h).  I guess it's again
  whatever subjproject code needs.

* Parts of the QAPI Schema subsystem

Aside: MAINTAINERS mostly not updated.

Your moves tear closely related code apart.  This is going to be a drag
for developers in general and maintainers in particular.

Ergonomics suffer when related code is in multiple places.  Having to
switch between directories and remember where is what will a constant
low-level pain.  Things that used to be simple & quick, like git-grep
qapi/*.c, become more involved.

Hurts even when merely consuming the subsystem: when I see #include
"qemu/foo.h", the straightforward include/qemu/foo.h may or may not do.
When it doesn't, I need to know where to look instead.

subprojects/qemu-common/include/ is a lot to type.  Sufficiently
powerful editors mitigate, but not completely.

When changes need to be applied to every instance of an abstraction,
it's easy to miss instances "elsewhere".  There's a reason the QAPI
visitors are all in one place.

The actual split seems somewhat arbitrary in places.  I suspect more
code will move over time.  Invalidating "what is where" knowledge.

I believe a serious think about other ways to accomplish your goals is
called for.

> (fwiw, it's a bit of a shame python-qemu-qmp didn't import git history from
> qemu.. we did better with libslirp. If we ever move code in standalone
> repositories again, we should be careful to keep history with it)

Yes, we should preserve history whenever practical.

[...]