Re: Making QEMU easier for management tools and applications

[Markus Armbruster]

Kevin Wolf <address@hidden> writes:

> Am 08.01.2020 um 12:40 hat Christophe de Dinechin geschrieben:
>> > On 8 Jan 2020, at 11:43, Kevin Wolf <address@hidden> wrote:
>> > Am 07.01.2020 um 18:11 hat Christophe de Dinechin geschrieben:
>> >> So I think that it might help, in the long run, to start defining the
>> >> language in question in some abstract way, and then to have rules
>> >> for how to transform that abstract language into concrete bindings.
>> > 
>> > I think this abstract language is QAPI.
>> 
>> Currently, yes, that’s the closest we have to a well-defined language,
>> I pointed it out in my mail, and there is qapi-code-gen.txt to prove it.
>
> Oh, did you? I must have missed that point, sorry.
>
>> So it would certainly make sense for the language I’m describing to
>> be a forward evolution of QAPI, or something that is trivially transformed
>> to / from the existing json schema.
>
> I guess I need to be more specific about what I mean with "QAPI". Of
> course, many things you can't currently do in QAPI because they are not
> implemented. When I'm talking about that QAPI should be used more, I'm
> mostly talking about applying its core concepts and infrastructure with
> the obvious extensions in some places, not about the existing
> implementation.
>
> So I guess "a forward evoluion of QAPI" is what I really have in mind.
>
>> > The problem is that we're not even close to using QAPI for everything.
>> 
>> That’s the symptom of the problem. The problem as I see it is that
>> we could not. at least not with today’s QAPI (maybe I’m wrong).
>> A good example is shortcut command-line options. I don’t think
>> you can, today, derive a user-friendly command-line syntax from
>> qapi-schema.json, and even less so one that is compatible with what
>> we have today.
>
> Well, you can't because it's not implemented. Implementing it from
> scratch without compatibility requirements would be relatively easy, and
> we're in fact going to try it out with the qemu-storage-daemon.

Yes.

Prior art:

    Presentation
    KVM Forum 2017: Towards a More Expressive and Introspectable QEMU
    Command Line
    https://www.youtube.com/watch?v=gtpOLQgnwug
    https://www.linux-kvm.org/images/f/f2/Armbru-qapi-cmdline_1.pdf

    RFC patches
    https://lists.nongnu.org/archive/html/qemu-devel/2017-10/msg00209.html
    Message-Id: <address@hidden>
    https://repo.or.cz/qemu/armbru.git/shortlog/refs/heads/qapi-cmdline

> But there is nothing in defining a command line option that would
> inherently be different in structure from defining a QMP command, so I
> would consider providing a command line option that is described by a
> QAPI type not a problem, even though it's not implemented yet.
>
> Compatibility is much harder, which is not the least because the
> existing command line options are bad (primarily: using inconsistent
> syntax).

Semantics are just as problematic.

>          The question is if we should make bad options work unchanged in
> the new infrastructure or if it wouldn't be better to replace them with
> something that behaves the same as the rest.
>
>> > Adding a new language on top of QAPI instead isn't going to make the
>> > conversion process any faster.
>> 
>> I fully agree that whatever language we select has to have an easy,
>> systematic forward path from QAPI, but also from the other
>> existing meta-languages, like the .def files used for options,

You mean the qemu-options.hx.  qemu-options.def is generated from it.

We used to have qmp-commands.hx, until QAPI swallowed it.  Command line
QAPIfication will swallow qemu-options.hx.

>> or build configuration (there is a hint of the connection between
>> the two in the option ‘if’ member in the .json files).

I'm not sure what exactly you have in mind for build configuration.

>> In other words, the language I have in mind would be something
>> that I could auto-generate from, say, the current qapi-schema.json
>> and the qemu-options.def. Maybe in the end, that would be simply
>> by having qemu-options.def being used to build qemu-options.json,
>> and then add the relevant entries in qemu-options.json. Maybe
>> there is a better way.
>
> I would rather say that .def should go away and its content should
> be converted to a JSON schema that becomes the new source format rather
> than an intermediate generated file. There is nothing in .def files that
> couldn't be represented in the schema.

Yes.

>> >> This definition itself is not obvious (at least not to me). For
>> >> example, do we have, anywhere but in the C code, the specification
>> >> of how one can add a disk to qemu, and what it means?
>> >> Say, looking at qemu-options.def, how do I tell that -hda has
>> >> anything to do with -device or -blockdev or -help?
>> > 
>> > BlockdevOptions in the QAPI schema is what tells you how it _really_
>> > works.
>> 
>> Not really, IMO, in the sense that it gives me no clue as to how -hda
>> or -device relate to it.
>> 
>> An important point that you raise through your response, though, is
>> that the QAPI schema is only an IDL (interface definition language).
>> In other words, it can tell me what a BlockdevOptions look like on
>> the wire, but at least in the current state, it cannot tell me what happens
>> with it.

True.

>> This is probably a good thing (it’s already complicated enough as is),
>> but it’s worth pointing out that, even sticking to a declarative language,
>> one could add constraints, as hinted by HAS_ARG in the .def file
>> (but we could have a richer constraints language, e.g. describing
>> -m option so that there can be at most one, the arg is a number,
>> can’t be negative or floating-point, and so on.
>
> To a certain degree, QAPI does just that, by supporting different data
> types for options. If necessary, I'm sure the type system could be
> extended, but it's not clear to me to which degree we actually need
> this.

QAPI is much more expressive than qemu-options.hx.  All the latter can
do is "has option argument" and "option is arch-specific".  QAPI lets
you specify an option argument's structure, and supports compile-time
conditionals.  It can't do things like "must specify either argument A
or B", or "numeric argument C must not exceed argument D", or "multiple
options combine vs. last one wins".  Mostly because "it’s already
complicated enough as is".

> Just drawing parallels from what we do for QMP commands, I imagine
> something like this to describe the -m option as it actually exists in
> qemu-options.def today:
>
> ##
> # @m:
> #
> # Configure guest RAM.
> #
> # @size: initial amount of guest memory
> # @slots: number of hotplug slots (default: none)
> # @maxmem: maximum amount of guest memory (default: none)
> ##
> { 'cmdline-option': 'm',
>   'arguments': {
>     'size': 'size',
>     'slots': '*int',
>     'maxmem': '*size'
>   },
>   'default-key': 'size',
>   'repeat': 'OVERRIDE' }
>
> HAS_ARG is automatically covered simply by 'arguments' being non-empty.
>
> (Markus probably already has a prototype with slightly different syntax,
> but the idea should be similar.)

A similar example from my RFC patches:

##
# @--add-fd:
# !texinfo
# -add-fd fd=@var{fd},set=@var{set}[,opaque=@var{opaque}]@*
#
# Add a file descriptor to an fd set.  Valid options are:
#
# @table @option
# @item fd=@var{fd}
# This option defines the file descriptor of which a duplicate is added to fd 
set.
# The file descriptor cannot be stdin, stdout, or stderr.
# @item set=@var{set}
# This option defines the ID of the fd set to add the file descriptor to.
# @item opaque=@var{opaque}
# This option defines a free-form string that can be used to describe @var{fd}.
# @end table
#
# You can open an image using pre-opened file descriptors from an fd set:
# @example
# qemu-system-i386
# -add-fd fd=3,set=2,opaque="rdwr:/path/to/file"
# -add-fd fd=4,set=2,opaque="rdonly:/path/to/file"
# -drive file=/dev/fdset/2,index=0,media=disk
# @end example
# !end texinfo
##
{ 'option': '--add-fd',
  'data': { 'fd': 'int', 'set': 'int', '*opaque': 'str' },
  'help': [
"-add-fd fd=fd,set=set[,opaque=opaque]",
"                Add 'fd' to fd 'set'"] }

>> And of course, we could decide to not stick to a declarative language,
>> but to have a way to have some, possibly limited, processing
>> described in the language. I’m thinking notably of what is called
>> “desugaring” in other threads.
>
> We could. But is it actually worth inventing a new programming language?
> I think this is something that should be done in C code even in the
> future. I think what we're looking for is a way to describe interfaces,
> not implementations.

Sane sugar can be described declaratively as macro expansion.

Less than sane sugar takes code.

Desugaring in code promotes (accidental) abandonment of sanity.

That said, we have to keep things simple to succeed.  Declarative
desugaring seems beyond our reach.

> Even just for adding more sophisticated constraints, like in the example
> above that maxmem >= size, it's questionable whether doing this in the
> schema provides enough value for actually implementing it there.

Yes.

>> >> A big issue, though, is that of compatibility. Doing the above starting
>> >> from scratch does not seem that complicated. Doing it in a way that
>> >> preserves a minimum of interoperability with earlier-generation
>> >> software is another ordeal.
>> > 
>> > Indeed, this is the major reason why QAPI isn't as pervasive as it
>> > should be.
>> 
>> Probably, but also because QAPI does not address the needs of
>> some of the things a slightly more general language could do.
>
> So what are the things that you would like to describe, but can't see an
> easy way to actually describe them with QAPI concepts?
>
>> >> So I think that Daniel is right. We may need at some point to start
>> >> a NEMU-style offshoot that does not attempt to be compatible,
>> >> but explores describing an increasing surface of the API using a
>> >> new meta-language from which we can generate, in a consistent
>> >> way, at least:
>> >> 
>> >> - C bindings
>> >> - Command-line options
>> >> - Shell bindings (or “HMP”)
>> >> - JSON schema or qom description
>> >> - Bindings in other languages (Rust, Go, Python)
>> >> - Networked versions of the API (socket, REST)
>> >> - Client-side code e.g. for libvirt.
>> >> - Serialization / deserialization, e.g. for configuration files
>> >> - Documentation, including man page and API docs
>> >> - Command-line help
>> > 
>> > I think the only thing in this list that can't obviously be covered
>> > easily by QAPI is QOM. Or rather, it's covered by passing through
>> > key=value lists without describing their structure
>> 
>> That’s close enough to me. (In my mind, that part was already “done”
>> by QAPI, even if in the convoluted way you describe)
>
> That's not really close enough. If you are happy with this level of
> abstraction, you can define any command line option to be either a flag
> or take a string as its argument and be done. This is obviously not very
> helpful because it says nothing about the structure of that string.

Similar to all QMP commands taking a single 'any' argument.

>                                                                     In
> the same way, QAPI can't say anything about the structure of a QOM
> object, and I think that's a problem.
>
>> > - which, as far as I
>> > understand, is mainly because QOM properties aren't necessarily static,
>> > so we can't provide a statically defined interface for them. Probably
>> > solvable in QEMU, but not without a major effort.
>> 
>> Or maybe extend the language so that it’s internal semantics
>> knows about this aspect of QOM?
>
> My point is that for example you can't generate a C struct (which is
> statically defined) from something that has a dynamic structure. The
> best you could do is fall back to key=value even in the C source, both
> key and value being strings. But then you still have to parse these
> strings manually, so it doesn't actually help much compared to a state
> without C bindings.

QOM and QAPI sabotage each other.  Ironic, considering they were dreamed
up by the same guy :)

QAPI is compile-time static by design.

QOM is run-time dynamic by design.  Some support for static definitions
has been grafted on since.

We use QAPI type system escapes for QOM.  Defeats QAPI introspection and
doc generation.  We provide separate QOM introspection instead, which is
clumsier and less expressive.  QOM documentation doesn't exist.

> Maybe what could be done is covering at least the static properties and
> then having key=value for the dynamic part (which should be the
> exception).

To make this worthwhile, we'd have to replace dynamic QOM properties by
static ones when possible.  Monumental task.

>> >> - Relations, e.g. how we represent “contains”, “derives from”, “needs”,
>> >> “one of”, “one or several”, “attaches to”…
>> >> - States, e.g. how do we represent the machine configuration,
>> >> or the desired new disk setting
>> >> - Verbs, e.g. how we represent “add”, “connect”, “remove”, “find”,
>> >> “start”, “notify”, etc. and how we describe the kind of input they need.
>> >> - Possibly more subtle things like support for transactions, 
>> >> commit/rollback,
>> >> i.e. “I want to add connect a virtual nic to some host vf, but if anything
>> >> along the way fails, I’d like all the cleanup to happen automatically)
>> > 
>> > This sounds like a different approach from our current QAPI command set
>> 
>> Well, except for purposefully trying to use a different wording to avoid
>> the risk of getting your mind stuck in one of the particular existing
>> meta-languages in QEMU, the approach is not very different.
>
> I didn't necessarily mean relations/state/verbs, which obviously exist,
> but the examples that seemed to express things in a deliberately
> different way from what we have today.
>
>> - Transactions do not exist today that I know of, although we see
>> signs of them in discussions about the fact that this options destroys
>> that back end but that option does not.
>
> We have a 'transaction' QMP command, but they are not an integral part
> of the language. Considering the trouble to implement transactional
> commands, I suppose we don't even want it to be a fundamental part of
> the language.
>
>> > Does it actually provide more functionality, though?
>> 
>> It’s not intended to provide more, but to require less to do the same thing.
>
> Though we always need to keep in mind that if requiring less for future
> additions requires a huge effort now, the investment may pay off only in
> a very distant future (if the abstraction we build even survives until
> then).

Worse is better.

http://dreamsongs.com/RiseOfWorseIsBetter.html