Re: Redesign of QEMU startup & initial configuration

[Paolo Bonzini]

On 12/9/21 20:11, Daniel P. Berrangé wrote:
> >     They still need to bootstrap a QMP monitor, and for that, CLI is fine
> >     as long as it's simple and stable.

I would go a step further and say that the QMP monitor socket should be 
created by whoever invoked QEMU and passed down via systemd's socket 
activation protocol, with a fallback to stdin/stdout.

> > 2. CLI and configuration files
> >
> >     Human users want a CLI and configuration files.
>
> I'd note that Kubernetes doesn't bother with a human CLI at
> all, and just expects everyone/everything to use JSON/YAML
> config files. So there's no divide between what syntax humans
> and machines use - humans just send the config via the CLI
> tool which uploads it to the REST API that machines use
> directly.

I agree with Mark here that configuration is beyond QEMU's role.  QEMU's 
JSON-based API isn't quite REST, but the idea is the same: the user 
talks to a tool which converts the configuration to QAPI commands.  The 
QAPI commands can be:

- invocations of qmp_* functions in the case of a command-line based 
configuration

- invocations of QMP commands on a socket if startup is mediated by a 
management tool

Where Markus and I disagree is whether the two cases should be covered 
by one binary or two.  Personally I don't see much advantage in having a 
single binary, just like I don't care if /bin/ls and /bin/dir are 
symlinks, or they're separate binaries, or one invokes the other.

What matters to me is the maintainability of the command-line based 
startup code.  Actual experience says that, as more and more options are 
supported via QMP-based configuration, the command-line based startup 
code gets simpler or at least more self-contained.

In the terms of my KVM Forum talk, this is obtained by converting 
"one-off" options (typically global variables) to a combination of a 
"shortcut" option (typically a qemu_opts_set or qemu_opts_parse 
invocation) and a QOM property or a QMP command.  Examples from recent 
QEMU versions include -no-shutdown, -no-hpet, -no-acpi.

> > = Ways forward =
>
> Just to clarify or remind ourselves of scope.
>
> When I think about startup/cli in QEMU, I essentially consider
> this to mean all softmmu/vl.c code.  Of course there are
> supporting pieces splattered around, but IMHO 90% of the pain
> is resulting from the code in the vl.c file.
>
> > 1. Improve what we have step by step, managing incompatible changes
> >
> >     We tried.  It has eaten substantial time and energy, it has
> >     complicated code and interfaces further, and we're still nowhere near
> >     where we need to be.  Does not feel like a practical way forward to
> >     me.
> >
> >     If you don't believe me, read the long version in "Appendix: Why
> >     further incremental change feels impractical".
>
> Essentially death by 1000 cuts.

There are two kinds of incremental changes.  Incremental changes to the 
UI are one thing; incremental changes to the implementations aka 
refactoring is another.

I agree that further incremental changes to the UI (in the style of 
-preconfig) are impractical, but incremental changes to the 
implementation are totally practical.  The reason for that is that there 
*is* a way forward for decoupling the options from the command line 
parsing code, and this way forward has *already* proved itself.

> Refering back to my earlier note, when I've suggested this approach
> in the past, I've basically considered it to mean stop touching
> softmmu/vl.c, and create a softmmu/vl2.c. The existing binaries
> remain using vl.c, we get legacy free binaries built from vl2.c
>
> Overtime we might get confidence to refactor bits of vl.c to
> reduce duplication, but I would consider that not a priority
> at least in the short term.

This is what I have in my tree, though it's called climain.c and 
qmpmain.c. :)  There is one major difference though: refactoring bits of 
vl.c to use more of QMP/QOM *is* the way you add functionality in qmpmain.c.

I disagree that vl.c is hard to refactor.  Sure there were some issues 
in 6.0/6.1, but at this point a lot of the hard parts have been done. 
The conversion of -smp to a QOM option, for example, was largely painless.

> On the other hand if we do a good job with the new binaries
> maybe there ceases to be any reason to keep the old binaries
> at all after a few years.

I'm not sure if this is possible if the new binaries provide exclusively 
a machine-oriented (JSON) interface.  But as you said, the pressure to 
keep a backwards-compatible command line would be much less with a good 
machine-oriented binary.

> > 3. Start over with the aim to replace the traditional QEMU
> >
> >     I don't want an end game where we have two QEMUs, one with bad CLI
> >     and limited QMP, and one without real CLI and decent QMP.  I want one
> >     QEMU with decent CLI and decent QMP.
>
> Right, so this is essentially what I've just suggested as a possible
> evoluation of (2). The only difference here is that you're setting
> the death of the old QEMU as an explicit end gaol in (3), while
> in (2) it is merely a nice-to-have.

I agree.  Though in my case I'd still go for two binaries: one with good 
CLI and limited QMP, and one with limited CLI and good QMP.

> In practical terms I don't think (2) and (3) are that different
> from each other for the first 2 years or so.  A lot can change in
> that time, so I don't think we need to fixate on a choice of
> (2) vs (3) upfront, just make it clear that the death of the old
> binaries is "on the table" as an outcome if we do well.

Agreed, but I would set the beginning of the "2 years or so" at last year...

> > = Appendix: Why further incremental change feels impractical =
> >
> > Crafting a big change in small steps sounds great.  It isn't when we
> > have to make things worse before they can get better, and every step is
> > painfully slow because it's just too hard, and the end state we aim for
> > isn't really what we want.
>
> I can't disagree with this. If we carry on trying to evolve vl.c
> incrementally we are doomed to be stuck with a horrible starstup
> process for enternity (or at least as long as I'll still be
> working as QEMU maintainer).

... and if you compare vl.c in 5.2 and now, and consider current vl.c to 
be horrible, my knowedge of English does not include an adjective to 
describe the 5.2 state.  Some incremental work _is_ possible or even 
necessary, and has been done already.

Paolo