[Qemu-devel] [PATCH v8 00/35] qapi visitor cleanups (post-introspection cleanups subset E)

[Eric Blake]

Based on qemu.git master. Pending prerequisites:
+ Not a strong dependency, but for qapi-tests to consistently pass,
I needed a race fixed:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01827.html

Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv8e

and will soon be part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi

v8 notes:
Four new patches (13-16/35), plus rebasing on top of them, so that
the code base now consistently passes a 'v, name' pair anywhere a
visitor needs a name, rather than putting other arguments in between
the pair. I got to have fun with Coccinelle :)  Also fix a bug in my
changes to visit_next_list() (v7 29/31), so that 'make check' and
qemu-iotests now pass at all points in the series.

The parameter ordering changes have the potential to be a rebase
magnet, so I'm hoping this series can go in relatively soon after
Markus returns from break.

I made good on my threat in v7 of writing a qapi-to-JSON output
visitor, but that will remain a separate series based on this one
(the only posting of that series so far now needs rebasing:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01760.html)

001/35:[----] [--] 'qobject: Document more shortcomings in our number handling'
002/35:[----] [--] 'qapi: Avoid use of misnamed DO_UPCAST()'
003/35:[----] [--] 'qapi: Drop dead dealloc visitor variable'
004/35:[----] [--] 'hmp: Improve use of qapi visitor'
005/35:[----] [--] 'vl: Improve use of qapi visitor'
006/35:[----] [--] 'balloon: Improve use of qapi visitor'
007/35:[----] [--] 'qapi: Improve generated event use of qapi visitor'
008/35:[----] [--] 'qapi: Track all failures between visit_start/stop'
009/35:[----] [--] 'qapi: Prefer type_int64 over type_int in visitors'
010/35:[----] [--] 'qapi: Make all visitors supply uint64 callbacks'
011/35:[----] [--] 'qapi: Consolidate visitor small integer callbacks'
012/35:[----] [--] 'qapi: Don't cast Enum* to int*'
013/35:[down] 'qom: Use typedef for Visitor'
014/35:[down] 'qapi: Swap visit_* arguments for consistent 'name' placement'
015/35:[down] 'qom: Swap 'name' next to visitor in ObjectPropertyAccessor'
016/35:[down] 'qapi: Swap 'name' in visit_* callbacks to match public API'
017/35:[0167] [FC] 'qapi: Drop unused 'kind' for struct/enum visit'
018/35:[----] [-C] 'qapi: Drop unused error argument for list and implicit 
struct'
019/35:[----] [-C] 'qmp: Fix reference-counting of qnull on empty output visit'
020/35:[----] [-C] 'qmp: Don't abuse stack to track qmp-output root'
021/35:[0027] [FC] 'qapi: Document visitor interfaces, add assertions'
022/35:[----] [-C] 'qapi: Add visit_type_null() visitor'
023/35:[0008] [FC] 'qmp: Tighten output visitor rules'
024/35:[0002] [FC] 'spapr_drc: Expose 'null' in qom-get when there is no fdt'
025/35:[----] [-C] 'qapi: Simplify excess input reporting in input visitors'
026/35:[----] [--] 'qapi: Add type.is_empty() helper'
027/35:[----] [--] 'qapi: Fix command with named empty argument type'
028/35:[----] [-C] 'qapi: Eliminate empty visit_type_FOO_fields'
029/35:[----] [-C] 'qapi: Canonicalize missing object to :empty'
030/35:[0006] [FC] 'qapi-visit: Unify struct and union visit'
031/35:[----] [-C] 'qapi: Rework deallocation of partial struct'
032/35:[----] [-C] 'qapi: Split visit_end_struct() into pieces'
033/35:[0015] [FC] 'qapi: Simplify semantics of visit_next_list()'
034/35:[0038] [FC] 'qapi: Change visit_type_FOO() to no longer return partial 
objects'
035/35:[----] [-C] 'RFC: qapi: Adjust layout of FooList types'

v7 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg01181.html
Rearrange several patches (trying to float the obvious ones up
front, moving hunks between patches to make review easier), and
add several new patches.  In particular, I'm quite pleased with
how 29/31 turned out as a new patch for simplifying visitor
semantics; and 18-20 address an issue that caused some last-minute
flurry before 2.5-rc3.

Patch 31 is marked RFC because it might be overkill; when I first
conceived it, I thought a simple change to qapi-types.py and
GenericList in visitor.h would be enough to save memory; until
valgrind reminded me that if sizeof(GenericList) shrinks and
becomes variably-sized, I have to start passing sizes around.  The
additional changes to visit_start_list() and visit_next_list()
didn't turn out as lean as I had hoped, so I'm okay if we ditch
that patch.

I'm very seriously considering writing a new JSON output visitor,
so that we can go straight from qapi to JSON without having to go
through an intermediate QObject tree, and as proof that the visitor
interface is starting to be better documented.  But I guess I should
first focus on getting my subset F patches ready to go (we still
want to get netdev_add introspectible), since those have at least
been on list before.

v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg05793.html
My set of patches related to qapi visitors has grown, and it's time
that I post it on list again.  Of course, since this is all 2.6
material, and there's already lots of patches earlier in the queue,
I may need a v7 to pick up rebase changes.

A lot of the new patches in this series are based on fallout from
implementing an early RFC posted against a v5 review:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06878.html

v5 and earlier - look in the mail archives

Eric Blake (35):
  qobject: Document more shortcomings in our number handling
  qapi: Avoid use of misnamed DO_UPCAST()
  qapi: Drop dead dealloc visitor variable
  hmp: Improve use of qapi visitor
  vl: Improve use of qapi visitor
  balloon: Improve use of qapi visitor
  qapi: Improve generated event use of qapi visitor
  qapi: Track all failures between visit_start/stop
  qapi: Prefer type_int64 over type_int in visitors
  qapi: Make all visitors supply uint64 callbacks
  qapi: Consolidate visitor small integer callbacks
  qapi: Don't cast Enum* to int*
  qom: Use typedef for Visitor
  qapi: Swap visit_* arguments for consistent 'name' placement
  qom: Swap 'name' next to visitor in ObjectPropertyAccessor
  qapi: Swap 'name' in visit_* callbacks to match public API
  qapi: Drop unused 'kind' for struct/enum visit
  qapi: Drop unused error argument for list and implicit struct
  qmp: Fix reference-counting of qnull on empty output visit
  qmp: Don't abuse stack to track qmp-output root
  qapi: Document visitor interfaces, add assertions
  qapi: Add visit_type_null() visitor
  qmp: Tighten output visitor rules
  spapr_drc: Expose 'null' in qom-get when there is no fdt
  qapi: Simplify excess input reporting in input visitors
  qapi: Add type.is_empty() helper
  qapi: Fix command with named empty argument type
  qapi: Eliminate empty visit_type_FOO_fields
  qapi: Canonicalize missing object to :empty
  qapi-visit: Unify struct and union visit
  qapi: Rework deallocation of partial struct
  qapi: Split visit_end_struct() into pieces
  qapi: Simplify semantics of visit_next_list()
  qapi: Change visit_type_FOO() to no longer return partial objects
  RFC: qapi: Adjust layout of FooList types

 backends/hostmem.c                      |  24 +--
 block/qapi.c                            |   2 +-
 blockdev.c                              |   4 +-
 bootdevice.c                            |  12 +-
 hmp.c                                   |  23 ++-
 hw/acpi/core.c                          |   4 +-
 hw/acpi/ich9.c                          |  49 ++---
 hw/block/nvme.c                         |  12 +-
 hw/core/machine.c                       |  24 +--
 hw/core/qdev-properties-system.c        |  44 ++--
 hw/core/qdev-properties.c               | 180 ++++++++--------
 hw/core/qdev.c                          |   7 +-
 hw/i386/pc.c                            |  43 ++--
 hw/ide/qdev.c                           |  12 +-
 hw/intc/xics.c                          |  20 +-
 hw/isa/lpc_ich9.c                       |   7 +-
 hw/mem/pc-dimm.c                        |   6 +-
 hw/misc/edu.c                           |   6 +-
 hw/misc/tmp105.c                        |  12 +-
 hw/net/ne2000-isa.c                     |  14 +-
 hw/pci-host/piix.c                      |  18 +-
 hw/pci-host/q35.c                       |  23 +--
 hw/ppc/spapr_drc.c                      |  43 ++--
 hw/usb/dev-storage.c                    |  12 +-
 hw/virtio/virtio-balloon.c              |  43 ++--
 include/qapi/qmp-output-visitor.h       |   1 +
 include/qapi/visitor-impl.h             | 106 ++++++----
 include/qapi/visitor.h                  | 298 ++++++++++++++++++++++++---
 include/qom/object.h                    |  13 +-
 memory.c                                |  26 +--
 net/dump.c                              |  12 +-
 net/filter-buffer.c                     |  14 +-
 net/net.c                               |   4 +-
 numa.c                                  |   6 +-
 qapi/opts-visitor.c                     | 121 ++++++-----
 qapi/qapi-dealloc-visitor.c             | 108 ++++------
 qapi/qapi-visit-core.c                  | 352 +++++++++++++++++---------------
 qapi/qmp-input-visitor.c                | 174 +++++++++-------
 qapi/qmp-output-visitor.c               | 157 +++++++-------
 qapi/string-input-visitor.c             |  93 +++++----
 qapi/string-output-visitor.c            |  88 ++++----
 qemu-img.c                              |  11 +-
 qobject/json-parser.c                   |   4 +-
 qobject/qjson.c                         |   8 +-
 qom/object.c                            | 129 ++++++------
 replay/replay-input.c                   |   4 +-
 scripts/qapi-commands.py                |  11 +-
 scripts/qapi-event.py                   |  24 +--
 scripts/qapi-types.py                   |  13 +-
 scripts/qapi-visit.py                   | 264 ++++++++++++------------
 scripts/qapi.py                         |  34 +--
 target-i386/cpu.c                       |  96 ++++-----
 target-ppc/translate_init.c             |  12 +-
 tests/qapi-schema/event-case.out        |   2 +-
 tests/qapi-schema/flat-union-empty.out  |   1 +
 tests/qapi-schema/ident-with-escape.out |   1 +
 tests/qapi-schema/indented-expr.out     |   4 +-
 tests/qapi-schema/qapi-schema-test.json |   2 +
 tests/qapi-schema/qapi-schema-test.out  |  47 ++++-
 tests/qapi-schema/union-clash-data.out  |   2 +
 tests/qapi-schema/union-empty.out       |   1 +
 tests/test-opts-visitor.c               |   6 +-
 tests/test-qdev-global-props.c          |  18 +-
 tests/test-qmp-commands.c               |  20 +-
 tests/test-qmp-input-strict.c           |  47 ++---
 tests/test-qmp-input-visitor.c          |  90 ++++----
 tests/test-qmp-output-visitor.c         |  38 ++--
 tests/test-string-input-visitor.c       |  34 +--
 tests/test-string-output-visitor.c      |  16 +-
 tests/test-visitor-serialization.c      |  54 ++---
 util/qemu-sockets.c                     |   4 +-
 vl.c                                    |  29 +--
 72 files changed, 1792 insertions(+), 1451 deletions(-)

-- 
2.4.3