[Qemu-devel] [PATCH v7 00/31] qapi visitor cleanups (post-introspection cleanups subset E)

[Eric Blake]

Pending prerequisites:
+ Markus' qapi-not-next branch (including my subset D patches)
http://repo.or.cz/qemu/armbru.git/shortlog/refs/heads/qapi-not-next
https://lists.gnu.org/archive/html/qemu-devel/2015-12/msg00463.html

Also available as a tag at this location:
git fetch git://repo.or.cz/qemu/ericb.git qapi-cleanupv7e

and will soon be part of my branch with the rest of the v5 series, at:
http://repo.or.cz/qemu/ericb.git/shortlog/refs/heads/qapi

v7 notes:
Rearrange several patches (trying to float the obvious ones up
front, moving hunks between patches to make review easier), and
add several new patches.  In particular, I'm quite pleased with
how 29/31 turned out as a new patch for simplifying visitor
semantics; and 18-20 address an issue that caused some last-minute
flurry before 2.5-rc3.

Patch 31 is marked RFC because it might be overkill; when I first
conceived it, I thought a simple change to qapi-types.py and
GenericList in visitor.h would be enough to save memory; until
valgrind reminded me that if sizeof(GenericList) shrinks and
becomes variably-sized, I have to start passing sizes around.  The
additional changes to visit_start_list() and visit_next_list()
didn't turn out as lean as I had hoped, so I'm okay if we ditch
that patch.

I'm very seriously considering writing a new JSON output visitor,
so that we can go straight from qapi to JSON without having to go
through an intermediate QObject tree, and as proof that the visitor
interface is starting to be better documented.  But I guess I should
first focus on getting my subset F patches ready to go (we still
want to get netdev_add introspectible), since those have at least
been on list before.

Some patch retitling makes the backport-diff claim more new patches
than there actually are.

001/31:[down] 'qobject: Document more shortcomings in our number handling'
002/31:[down] 'qapi: Avoid use of misnamed DO_UPCAST()'
003/31:[down] 'qapi: Drop dead dealloc visitor variable'
004/31:[0008] [FC] 'hmp: Improve use of qapi visitor'
005/31:[0012] [FC] 'vl: Improve use of qapi visitor'
006/31:[----] [--] 'balloon: Improve use of qapi visitor'
007/31:[0007] [FC] 'qapi: Improve generated event use of qapi visitor'
008/31:[----] [--] 'qapi: Track all failures between visit_start/stop'
009/31:[down] 'qapi: Prefer type_int64 over type_int in visitors'
010/31:[down] 'qapi: Make all visitors supply uint64 callbacks'
011/31:[down] 'qapi: Consolidate visitor small integer callbacks'
012/31:[0002] [FC] 'qapi: Don't cast Enum* to int*'
013/31:[down] 'qapi: Drop unused 'kind' for struct/enum visit'
014/31:[0013] [FC] 'qapi: Drop unused error argument for list and implicit 
struct'
015/31:[0021] [FC] 'qmp: Fix reference-counting of qnull on empty output visit'
016/31:[down] 'qmp: Don't abuse stack to track qmp-output root'
017/31:[down] 'qapi: Document visitor interfaces, add assertions'
018/31:[down] 'qapi: Add visit_type_null() visitor'
019/31:[down] 'qmp: Tighten output visitor rules'
020/31:[down] 'spapr_drc: Expose 'null' in qom-get when there is no fdt'
021/31:[down] 'qapi: Simplify excess input reporting in input visitors'
022/31:[----] [-C] 'qapi: Add type.is_empty() helper'
023/31:[----] [--] 'qapi: Fix command with named empty argument type'
024/31:[0020] [FC] 'qapi: Eliminate empty visit_type_FOO_fields'
025/31:[0002] [FC] 'qapi: Canonicalize missing object to :empty'
026/31:[0002] [FC] 'qapi-visit: Unify struct and union visit'
027/31:[0002] [FC] 'qapi: Rework deallocation of partial struct'
028/31:[0030] [FC] 'qapi: Split visit_end_struct() into pieces'
029/31:[down] 'qapi: Simplify semantics of visit_next_list()'
030/31:[0154] [FC] 'qapi: Change visit_type_FOO() to no longer return partial 
objects'
031/31:[down] 'RFC: qapi: Adjust layout of FooList types

v6 notes:
https://lists.gnu.org/archive/html/qemu-devel/2015-11/msg05793.html
My set of patches related to qapi visitors has grown, and it's time
that I post it on list again.  Of course, since this is all 2.6
material, and there's already lots of patches earlier in the queue,
I may need a v7 to pick up rebase changes.

A lot of the new patches in this series are based on fallout from
implementing an early RFC posted against a v5 review:
https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg06878.html

v5 and earlier - look in the mail archives :)

Eric Blake (31):
  qobject: Document more shortcomings in our number handling
  qapi: Avoid use of misnamed DO_UPCAST()
  qapi: Drop dead dealloc visitor variable
  hmp: Improve use of qapi visitor
  vl: Improve use of qapi visitor
  balloon: Improve use of qapi visitor
  qapi: Improve generated event use of qapi visitor
  qapi: Track all failures between visit_start/stop
  qapi: Prefer type_int64 over type_int in visitors
  qapi: Make all visitors supply uint64 callbacks
  qapi: Consolidate visitor small integer callbacks
  qapi: Don't cast Enum* to int*
  qapi: Drop unused 'kind' for struct/enum visit
  qapi: Drop unused error argument for list and implicit struct
  qmp: Fix reference-counting of qnull on empty output visit
  qmp: Don't abuse stack to track qmp-output root
  qapi: Document visitor interfaces, add assertions
  qapi: Add visit_type_null() visitor
  qmp: Tighten output visitor rules
  spapr_drc: Expose 'null' in qom-get when there is no fdt
  qapi: Simplify excess input reporting in input visitors
  qapi: Add type.is_empty() helper
  qapi: Fix command with named empty argument type
  qapi: Eliminate empty visit_type_FOO_fields
  qapi: Canonicalize missing object to :empty
  qapi-visit: Unify struct and union visit
  qapi: Rework deallocation of partial struct
  qapi: Split visit_end_struct() into pieces
  qapi: Simplify semantics of visit_next_list()
  qapi: Change visit_type_FOO() to no longer return partial objects
  RFC: qapi: Adjust layout of FooList types

 hmp.c                                   |  19 +-
 hw/core/qdev-properties.c               |   6 +-
 hw/ppc/spapr_drc.c                      |  19 +-
 hw/virtio/virtio-balloon.c              |  25 +--
 include/qapi/qmp-output-visitor.h       |   1 +
 include/qapi/visitor-impl.h             |  92 ++++++----
 include/qapi/visitor.h                  | 281 ++++++++++++++++++++++++++---
 qapi/opts-visitor.c                     | 111 +++++++-----
 qapi/qapi-dealloc-visitor.c             |  97 ++++------
 qapi/qapi-visit-core.c                  | 309 +++++++++++++++++---------------
 qapi/qmp-input-visitor.c                | 158 ++++++++--------
 qapi/qmp-output-visitor.c               | 149 +++++++--------
 qapi/string-input-visitor.c             |  83 +++++----
 qapi/string-output-visitor.c            |  78 ++++----
 qobject/json-parser.c                   |   4 +-
 qobject/qjson.c                         |   8 +-
 qom/object.c                            |  13 +-
 scripts/qapi-commands.py                |   7 +-
 scripts/qapi-event.py                   |  24 +--
 scripts/qapi-types.py                   |  11 +-
 scripts/qapi-visit.py                   | 248 +++++++++++++------------
 scripts/qapi.py                         |  30 ++--
 tests/qapi-schema/event-case.out        |   2 +-
 tests/qapi-schema/flat-union-empty.out  |   1 +
 tests/qapi-schema/ident-with-escape.out |   1 +
 tests/qapi-schema/indented-expr.out     |   4 +-
 tests/qapi-schema/qapi-schema-test.json |   2 +
 tests/qapi-schema/qapi-schema-test.out  |  47 ++++-
 tests/qapi-schema/union-clash-data.out  |   2 +
 tests/qapi-schema/union-empty.out       |   1 +
 tests/test-qmp-commands.c               |  18 +-
 tests/test-qmp-input-strict.c           |  19 +-
 tests/test-qmp-input-visitor.c          |  10 +-
 tests/test-qmp-output-visitor.c         |   4 +
 vl.c                                    |  29 +--
 35 files changed, 1129 insertions(+), 784 deletions(-)

-- 
2.4.3