Re: Making QEMU easier for management tools and applications

[Daniel P . Berrangé]

On Fri, Jan 24, 2020 at 08:59:41AM +0100, Markus Armbruster wrote:
> John Snow <address@hidden> writes:
> 
> > On 1/23/20 2:01 PM, Daniel P. Berrangé wrote:
> >> So when configuring objects you'll always provide a JSON/YAML doc.
> >> They've got some clever stuff for updating objects where you can
> >> provide a JSON patch for only the bits which need changing.
> >> 
> >> When querying/listing objects by default it displays only a small
> >> subset of their config information in a human friendly-ish format.
> >> If you want to see everything then you ask for it in JSON/YAML
> >> format. There's also an expression language that lets you extract
> >> particular pieces of information based on requested properties,
> >> and you can filter the list of objects based on attributes and so
> >> on.
> >> 
> >> I think it is fair to say the structure of kubernetes object config
> >> is on a par with hierarchical complexity of QEMU. The lack of a simple
> >> human targetted data input format does not appear to have negatively
> >> impacted the adoption of Kubernetes. It is worth questioning why this
> >> is the case, while we feel the human CLI syntax for QEMU is so
> >> critically important to QEMU's future ?
> 
> I consider human CLI syntax for QEMU a mostly solved *design* problem:
> dotted keys.  It's an unsolved *implementation* problem: the CLI is a
> tangled mess of almost two decades' worth of ideas, and only (some of)
> the latest strands actually use dotted keys infrastructure.  The
> proposed solution is CLI QAPIfication.  Gives us configuration file(s)
> and introspection.
> 
> Dotted keys are merely yet another concrete syntax.  They're designed to
> satisfy the CLI requirements we have, which include a measure of
> compatibility to what's in the tangled mess.  They're reasonably usable
> for simple stuff, but complex stuff can be too verbose to be readable.
> They can't express all of the abstract syntax.  Tolerable, since they
> provide an escape to JSON.  I recommend programs use the JSON escape
> always.  Awkward for humans due to shell quoting.

I agree that the dotted key syntax is our chosen / solved design
for expressing JSON on the CLI. I would also say that, in retrospect,
this was a incorrect design decision that is one of the key things
responsible for QEMU having a bad reputation for complexity.

We should simply never have tried to invent a way to map the full
hiearchy of JSON onto the CLI as the result will always be unpleasant.
The dotted notation is the most verbose way to do this type of
configuration, because of the string repetition it requires for
nested structures.

Lets consider how libvirt uses blockdev for a LUKS volume stored
in iSCSI

  $ qemu-system-x86_64 \
  -object secret,id=libvirt-5-storage-secret0,\
    data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
    keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
  -object secret,id=libvirt-5-format-luks-secret0,\
    data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
    keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
  -blockdev '{"driver":"iscsi","portal":"example.org:6000",\
    "target":"iqn.1992-01.com.example:storage","lun":1,"transport":"tcp",\
    "user":"myname","password-secret":"libvirt-5-storage-secret0",\
    "node-name":"libvirt-5-storage","auto-read-only":true,"discard":"unmap"}' \
  -blockdev 
'{"node-name":"libvirt-5-format","read-only":false,"driver":"qcow2",\
    "encrypt":{"format":"luks","key-secret":"libvirt-5-format-luks-secret0"},\
    "file":"libvirt-5-storage"}' \

We all know JSON is horrible on the CLI, no surprise. So

Lets use human "friendly" dotted syntax instead:

  $ qemu-system-x86_64 \
  -object secret,id=libvirt-5-storage-secret0,\
    data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
    keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
  -object secret,id=libvirt-5-format-luks-secret0,\
    data=9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1,\
    keyid=masterKey0,iv=AAECAwQFBgcICQoLDA0ODw==,format=base64 \
  -blockdev driver=qcow2,node-name=libvirt-5-format,read-only=false,\
    encrypt.format=luks,encrypt.key-secret=libvirt-5-format-luks-secret0,\
    file.driver=iscsi,file.portal=example.org:6000,\
    file.target=iqn.1992-01.com.example:storage,file.lun=1,file.transport=tcp,\
    file.user=myname,file.password-secret=libvirt-6-storage-secret0,\
    file.node-name=libvirt-5-storage,file.auto-read-only=true,file.dicard=unmap

I don't think that's much of an improvement, aside from not having
to worry about matching "}".

If we move to JSON in a config file

  $ cat qemu.json
  {
    "arguments": [
      {
        "arg": "object",
        "data": {
          "type": "secret",
          "id":"libvirt-5-storage-secret0",
          "data": 
"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1",
          "keyid": "masterKey0",
          "iv": "AAECAwQFBgcICQoLDA0ODw==",
          "format": "base64"
        }
      },
      {
        "arg": "object",
        "data": {
          "type": "secret",
          "id":"libvirt-5-format-luks-secret0",
          "data": 
"9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1",
          "keyid": "masterKey0",
          "iv": "AAECAwQFBgcICQoLDA0ODw==",
          "format": "base64"
        }
      },
      {
        "arg": "blockdev",
        "data": {
          "node-name":"libvirt-5-format",
          "read-only":false,"driver":"qcow2",
          "encrypt":{
            "format":"luks","key-secret":
            "libvirt-5-format-luks-secret0"
          },
          "file":{
            "driver": "iscsi",
            "portal": "example.org:6000",
            "target":"iqn.1992-01.com.example:storage",
            "lun": 1,
            "transport": "tcp",
            "user": "myname",
            "password-secret": "libvirt-5-storage-secret0",
            "node-name":"libvirt-5-storage",
            "auto-read-only":"true",
            "discard":"unmap"
          }
        }
      }
    ]
  }
  $ qemu-system-x86_64 -f qemu.json

The config file is more volumous than the CLI, but it is also
massively more intelligible to humans because you can see the
structure of the data.

I still screwed up many times with missing quotes, incorrect
commas, etc. All the fun of JSON

So if we allowed YAML instead of JSON, now we get...

  $ cat qemu.yaml
  ---
  arguments:
  - arg: object
    data:
      type: secret
      id: libvirt-5-storage-secret0
      data: 9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1
      keyid: masterKey0
      iv: AAECAwQFBgcICQoLDA0ODw==
      format: base64
  - arg: object
    data:
      type: secret
      id: libvirt-5-format-luks-secret0
      data: 9eao5F8qtkGt+seB1HYivWIxbtwUu6MQtg1zpj/oDtUsPr1q8wBYM91uEHCn6j/1
      keyid: masterKey0
      iv: AAECAwQFBgcICQoLDA0ODw==
      format: base64
  - arg: blockdev
    data:
      node-name: libvirt-5-format
      read-only: false
      driver: qcow2
      encrypt:
        format: luks
        key-secret: libvirt-5-format-luks-secret0
      file:
        driver: iscsi
        portal: example.org:6000
        target: iqn.1992-01.com.example:storage
        lun: 1
        transport: tcp
        user: myname
        password-secret: libvirt-5-storage-secret0
        node-name: libvirt-5-storage
        auto-read-only: true
        discard: unmap
  $ qemu-system-x86_64 -f qemu.yaml

This is finally something I'd consider to be on a par with the
original QEMU syntax, before we added hierarchical data. You
have the minimal possible amount of syntax here. No commas,
no quotes, no curly brackets, etc.

I'm sure there are things about YAML that are not nice, but
at some point there's a tradeoff where we can declare it
"good enough". You mention that it has a huge design spec,
but does that really matter ? We shouldn't be implemneting
a parser or formatter ourselves, just re-use one that
already exists. What matters more is whether it is nice
for humans & machines, and I think it is a good syntax
in both those cases.


> > However, the docs being badly out of date are a problem. We actively
> > lead people towards harmful / difficult to support paradigms.
> >
> >> All this results in a situation where JSON is functionally the best
> >> way to configure QEMU, but practically the worse, since very few
> >> people understand how to actually use it. This is a vicious circle
> >> holding back QMP/JSON and making the human syntax an ever greater
> >> burden for users & maintainers
> >> 
> >
> > I do basically agree.
> 
> What to do about it?
> 
> Here's an idea that hasn't been mentioned in this thread: a -writeconfig
> that actually works.  After you configured QEMU in whatever ways you
> rustled up on the 'net, you can -writeconfig the resulting command line
> into a *modern* configuration file.

No matter what we do implementation / designwise, the pre-requisite for
all of them is that we're able to fully express the configuration of the
VM using QAPI. As a side effect this means that -loadconfig and -writeconfig
become usable in the real world, which is nice for the current users of
qemu-system-XXX.  Per my reply to John though, I think we could consider
being more ambituous & seek to obslete qemu-system-XXX entirely at some
point in the future.

> >> IOW, the difficulty with configuring QEMU via JSON is not the fault
> >> of JSON itself, it is the lack of knowledge amongst users and docs,
> >> compounded by our never ending "improvements" to the human syntax.
> >> There are other factors too, such as our only partial coverage of
> >> config using JSON - some is only possible via the CLI still.
> >
> > I'm fine with getting rid of HMP entirely, I think. It's a weird
> > interface with bizarre behavior that's hard to support.
> >
> > There's a few commands in there we just don't support at all, but maybe
> > it's time to start deprecating one-by-one any of the individual commands
> > that are better served by QMP these days, to send the message that HMP's
> > days are numbered.
> >
> > Bye-bye!
> 
> Experience tells that no matter how weird and bizarre a feature is, once
> you attempt to remove it, it *will* find champions willing to fight for
> it to the death.  I'm not trying to tell you "don't go there", only "if
> you go there, go armed and prepared".

Yeah and it gets worse the older / more mature a project is. This
is what pushes projects into doing a clean break "2.0" design,
which is something we've never done for QEMU.  I don't mean just
breaking backcompat in a handful of CLI args here. I mean a proper
scorched earth, everything is up for re-evaluation, "2.0".


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|