Re: [Qemu-devel] [PATCH v5 03/11] qom: support arbitrary non-scalar properties with -object

[Markus Armbruster]

"Daniel P. Berrange" <address@hidden> writes:

> The current -object command line syntax only allows for
> creation of objects with scalar properties, or a list
> with a fixed scalar element type. Objects which have
> properties that are represented as structs in the QAPI
> schema cannot be created using -object.
>
> This is a design limitation of the way the OptsVisitor
> is written. It simply iterates over the QemuOpts values
> as a flat list. The support for lists is enabled by
> allowing the same key to be repeated in the opts string.
>
> It is not practical to extend the OptsVisitor to support
> more complex data structures while also maintaining
> the existing list handling behaviour that is relied upon
> by other areas of QEMU.

It should be practical to create a new option input visitor that parses
command line syntax straight into a QAPI visit, without the list magic,
and with fewer or no restrictions.  Then we could start defining complex
command line arguments as QAPI types, parse them straight into generated
QAPI types, and dispense with the QDict wrangling.

But I can't blame you for solving a simpler problem instead.  In
particular since there's ample precedence for QDict wrangling.

> Fortunately there is no existing object that implements
> the UserCreatable interface that relies on the list
> handling behaviour, so it is possible to swap out the
> OptsVisitor for a different visitor implementation, so
> -object supports non-scalar properties, thus leaving
> other users of OptsVisitor unaffected.
>
> The previously added qdict_crumple() method is able to
> take a qdict containing a flat set of properties and
> turn that into a arbitrarily nested set of dicts and
> lists. By combining qemu_opts_to_qdict and qdict_crumple()
> together, we can turn the opt string into a data structure
> that is practically identical to that passed over QMP
> when defining an object. The only difference is that all
> the scalar values are represented as strings, rather than
> strings, ints and bools. This is sufficient to let us
> replace the OptsVisitor with the QMPInputVisitor for
> use with -object.
>
> Thus -object can now support non-scalar properties,
> for example the QMP object
>
>   {
>     "execute": "object-add",
>     "arguments": {
>       "qom-type": "demo",
>       "id": "demo0",
>       "parameters": {
>         "foo": [
>           { "bar": "one", "wizz": "1" },
>           { "bar": "two", "wizz": "2" }
>         ]
>       }
>     }
>   }
>
> Would be creatable via the CLI now using
>
>     $QEMU \
>       -object demo,id=demo0,\
>               foo.0.bar=one,foo.0.wizz=1,\
>               foo.1.bar=two,foo.1.wizz=2
>
> Notice that this syntax is intentionally compatible
> with that currently used by block drivers.
>
> This is also wired up to work for the 'object_add' command
> in the HMP monitor with the same syntax.
>
>   (hmp) object_add demo,id=demo0,\
>                    foo.0.bar=one,foo.0.wizz=1,\
>                    foo.1.bar=two,foo.1.wizz=2
>
> NB indentation should not be used with HMP commands, this
> is just for convenient formatting in this commit message.

Cannot be used, actually (and neither can line continuation), but I get
what you mean.

> Signed-off-by: Daniel P. Berrange <address@hidden>
> ---
>  hmp.c                           |  18 +--
>  include/qom/object_interfaces.h |  10 +-
>  qmp.c                           |   2 +-
>  qom/object_interfaces.c         |  49 +++++--
>  tests/check-qom-proplist.c      | 317 
> +++++++++++++++++++++++++++++++++++++++-
>  5 files changed, 363 insertions(+), 33 deletions(-)
>
> diff --git a/hmp.c b/hmp.c
> index a4b1d3d..1972bef 100644
> --- a/hmp.c
> +++ b/hmp.c
> @@ -25,7 +25,7 @@
>  #include "qemu/sockets.h"
>  #include "monitor/monitor.h"
>  #include "monitor/qdev.h"
> -#include "qapi/opts-visitor.h"
> +#include "qapi/qmp-input-visitor.h"
>  #include "qapi/qmp/qerror.h"
>  #include "qapi/string-output-visitor.h"
>  #include "qapi/util.h"
> @@ -1717,20 +1717,12 @@ void hmp_netdev_del(Monitor *mon, const QDict *qdict)
>  void hmp_object_add(Monitor *mon, const QDict *qdict)
>  {
>      Error *err = NULL;
> -    QemuOpts *opts;
> -    OptsVisitor *ov;
> +    QmpInputVisitor *qiv;
>      Object *obj = NULL;
>  
> -    opts = qemu_opts_from_qdict(qemu_find_opts("object"), qdict, &err);
> -    if (err) {
> -        hmp_handle_error(mon, &err);
> -        return;
> -    }
> -
> -    ov = opts_visitor_new(opts);
> -    obj = user_creatable_add(qdict, opts_get_visitor(ov), &err);
> -    opts_visitor_cleanup(ov);
> -    qemu_opts_del(opts);
> +    qiv = qmp_input_visitor_new((QObject *)qdict, true, true);
> +    obj = user_creatable_add(qdict, qmp_input_get_visitor(qiv), &err);
> +    qmp_input_visitor_cleanup(qiv);

Any patch that gets rid of a qemu_opts_from_qdict() deserves support.

>  
>      if (err) {
>          hmp_handle_error(mon, &err);
> diff --git a/include/qom/object_interfaces.h b/include/qom/object_interfaces.h
> index 8b17f4d..997563a 100644
> --- a/include/qom/object_interfaces.h
> +++ b/include/qom/object_interfaces.h
> @@ -96,18 +96,24 @@ Object *user_creatable_add(const QDict *qdict,
>   * user_creatable_add_type:
>   * @type: the object type name
>   * @id: the unique ID for the object
> + * @nested: whether to recurse into the visitor for properties
>   * @qdict: the object properties
>   * @v: the visitor
>   * @errp: if an error occurs, a pointer to an area to store the error
>   *
>   * Create an instance of the user creatable object @type, placing
>   * it in the object composition tree with name @id, initializing
> - * it with properties from @qdict
> + * it with properties from @qdict.
> + *
> + * If the visitor is already positioned to read the properties
> + * in @qdict, @nested should be false. Conversely, if it is
> + * neccessary to open/close a struct to read the properties in

necessary

> + * @qdict, @nested should be true.

I don't like this interface much, but I'm not the maintainer here.  Have
you considered creating a function that does nested=false and can be
used by the nested=true function?

>   *
>   * Returns: the newly created object or NULL on error
>   */
>  Object *user_creatable_add_type(const char *type, const char *id,
> -                                const QDict *qdict,
> +                                bool nested, const QDict *qdict,
>                                  Visitor *v, Error **errp);
>  
>  /**
> diff --git a/qmp.c b/qmp.c
> index dcd0953..dc5fff0 100644
> --- a/qmp.c
> +++ b/qmp.c
> @@ -666,7 +666,7 @@ void qmp_object_add(const char *type, const char *id,
>      }
>  
>      qiv = qmp_input_visitor_new(props, true, false);
> -    obj = user_creatable_add_type(type, id, pdict,
> +    obj = user_creatable_add_type(type, id, true, pdict,
>                                    qmp_input_get_visitor(qiv), errp);
>      qmp_input_visitor_cleanup(qiv);
>      if (obj) {
> diff --git a/qom/object_interfaces.c b/qom/object_interfaces.c
> index 51e62e2..0883c91 100644
> --- a/qom/object_interfaces.c
> +++ b/qom/object_interfaces.c
> @@ -3,6 +3,7 @@
>  #include "qom/object_interfaces.h"
>  #include "qemu/module.h"
>  #include "qapi-visit.h"
> +#include "qapi/qmp-input-visitor.h"
>  #include "qapi/qmp-output-visitor.h"
>  #include "qapi/opts-visitor.h"
>  
> @@ -63,12 +64,16 @@ Object *user_creatable_add(const QDict *qdict,
>      if (local_err) {
>          goto out_visit;
>      }
> -    visit_check_struct(v, &local_err);
> +
> +    obj = user_creatable_add_type(type, id, false, pdict, v, &local_err);
>      if (local_err) {
>          goto out_visit;
>      }
>  
> -    obj = user_creatable_add_type(type, id, pdict, v, &local_err);
> +    visit_check_struct(v, &local_err);
> +    if (local_err) {
> +        goto out_visit;
> +    }
>  
>  out_visit:
>      visit_end_struct(v);
> @@ -87,7 +92,7 @@ out:
>  
>  
>  Object *user_creatable_add_type(const char *type, const char *id,
> -                                const QDict *qdict,
> +                                bool nested, const QDict *qdict,
>                                  Visitor *v, Error **errp)
>  {
>      Object *obj;
> @@ -114,9 +119,11 @@ Object *user_creatable_add_type(const char *type, const 
> char *id,
>  
>      assert(qdict);
>      obj = object_new(type);
> -    visit_start_struct(v, NULL, NULL, 0, &local_err);
> -    if (local_err) {
> -        goto out;
> +    if (nested) {
> +        visit_start_struct(v, NULL, NULL, 0, &local_err);
> +        if (local_err) {
> +            goto out;
> +        }
>      }
>      for (e = qdict_first(qdict); e; e = qdict_next(qdict, e)) {
>          object_property_set(obj, v, e->key, &local_err);
> @@ -124,12 +131,14 @@ Object *user_creatable_add_type(const char *type, const 
> char *id,
>              break;
>          }
>      }
> -    if (!local_err) {
> -        visit_check_struct(v, &local_err);
> -    }
> -    visit_end_struct(v);
> -    if (local_err) {
> -        goto out;
> +    if (nested) {
> +        if (!local_err) {
> +            visit_check_struct(v, &local_err);
> +        }
> +        visit_end_struct(v);
> +        if (local_err) {
> +            goto out;
> +        }
>      }
>  
>      object_property_add_child(object_get_objects_root(),
> @@ -156,15 +165,23 @@ out:
>  
>  Object *user_creatable_add_opts(QemuOpts *opts, Error **errp)
>  {
> -    OptsVisitor *ov;
> +    QmpInputVisitor *qiv;
>      QDict *pdict;
> +    QObject *pobj;
>      Object *obj = NULL;
>  
> -    ov = opts_visitor_new(opts);
>      pdict = qemu_opts_to_qdict(opts, NULL);
>  
> -    obj = user_creatable_add(pdict, opts_get_visitor(ov), errp);
> -    opts_visitor_cleanup(ov);
> +    pobj = qdict_crumple(pdict, true, errp);
> +    if (!pobj) {
> +        goto cleanup;
> +    }
> +    qiv = qmp_input_visitor_new(pobj, true, true);
> +
> +    obj = user_creatable_add((QDict *)pobj, qmp_input_get_visitor(qiv), 
> errp);
> +    qmp_input_visitor_cleanup(qiv);
> +    qobject_decref(pobj);
> + cleanup:
>      QDECREF(pdict);
>      return obj;
>  }
[...]

I'm not familiar enough with this code to provide a real review with
reasonable effort.  It looks sane to ignorant me.