[Qemu-devel] [PULL 10/12] qapi: Lazy creation of array types

[Markus Armbruster]

From: Eric Blake <address@hidden>

Commit ac88219a had several TODO markers about whether we needed
to automatically create the corresponding array type alongside
any other type.  It turns out that most of the time, we don't!

There are a few exceptions: 1) We have a few situations where we
use an array type in internal code but do not expose that type
through QMP; fix it by declaring a dummy type that forces the
generator to see that we want to use the array type.

2) The builtin arrays (such as intList for QAPI ['int']) must
always be generated, because of the way our QAPI_TYPES_BUILTIN
compile guard works: we have situations (at the very least
tests/test-qmp-output-visitor.c) that include both top-level
"qapi-types.h" (via "error.h") and a secondary
"test-qapi-types.h". If we were to only emit the builtin types
when used locally, then the first .h file would not include all
types, but the second .h does not declare anything at all because
the first .h set QAPI_TYPES_BUILTIN, and we would end up with
compilation error due to things like unknown type 'int8List'.

Actually, we may need to revisit how we do type guards, and
change from a single QAPI_TYPES_BUILTIN over to a different
usage pattern that does one #ifdef per qapi type - right now,
the only types that are declared multiple times between two qapi
.json files for inclusion by a single .c file happen to be the
builtin arrays.  But now that we have QAPI 'include' statements,
it is logical to assume that we will soon reach a point where
we want to reuse non-builtin types (yes, I'm thinking about what
it will take to add introspection to QGA, where we will want to
reuse the SchemaInfo type and friends).  One #ifdef per type
will help ensure that generating the same qapi type into more
than one qapi-types.h won't cause collisions when both are
included in the same .c file; but we also have to solve how to
avoid creating duplicate qapi-types.c entry points.  So that
is a problem left for another day.

Generated code for qapi-types and qapi-visit is drastically
reduced; less than a third of the arrays that were blindly
created were actually needed (a quick grep shows we dropped
from 219 to 69 *List types), and the .o files lost more than
30% of their bulk.  [For best results, diff the generated
files with 'git diff --patience --no-index pre post'.]

Interestingly, the introspection output is unchanged - this is
because we already cull all types that are not indirectly
reachable from a command or event, so introspection was already
using only a subset of array types.  The subset of types
introspected is now a much larger percentage of the overall set
of array types emitted in qapi-types.h (since the larger set
shrunk), but still not 100% (evidence that the array types
emitted for our new Dummy structs, and the new struct itself,
don't affect QMP).

Signed-off-by: Eric Blake <address@hidden>
Message-Id: <address@hidden>
[Moved array info tracking to a later patch]
Signed-off-by: Markus Armbruster <address@hidden>
---
 qapi-schema.json                        | 11 +++++++++++
 scripts/qapi.py                         | 11 ++++++-----
 tests/qapi-schema/qapi-schema-test.json |  4 ++++
 tests/qapi-schema/qapi-schema-test.out  |  3 +++
 4 files changed, 24 insertions(+), 5 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index a386605..702b7b5 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3416,6 +3416,17 @@
             'features': 'int' } }
 
 ##
+# @DummyForceArrays
+#
+# Not used by QMP; hack to let us use X86CPUFeatureWordInfoList internally
+#
+# Since 2.5
+##
+{ 'struct': 'DummyForceArrays',
+  'data': { 'unused': ['X86CPUFeatureWordInfo'] } }
+
+
+##
 # @RxState:
 #
 # Packets receiving state
diff --git a/scripts/qapi.py b/scripts/qapi.py
index d7cf0f3..9e01705 100644
--- a/scripts/qapi.py
+++ b/scripts/qapi.py
@@ -1143,7 +1143,12 @@ class QAPISchema(object):
     def _def_builtin_type(self, name, json_type, c_type, c_null):
         self._def_entity(QAPISchemaBuiltinType(name, json_type,
                                                c_type, c_null))
-        self._make_array_type(name)     # TODO really needed?
+        # TODO As long as we have QAPI_TYPES_BUILTIN to share multiple
+        # qapi-types.h from a single .c, all arrays of builtins must be
+        # declared in the first file whether or not they are used.  Nicer
+        # would be to use lazy instantiation, while figuring out how to
+        # avoid compilation issues with multiple qapi-types.h.
+        self._make_array_type(name)
 
     def _def_predefineds(self):
         for t in [('str',    'string',  'char' + pointer_suffix, 'NULL'),
@@ -1192,7 +1197,6 @@ class QAPISchema(object):
         data = expr['data']
         prefix = expr.get('prefix')
         self._def_entity(QAPISchemaEnumType(name, info, data, prefix))
-        self._make_array_type(name)     # TODO really needed?
 
     def _make_member(self, name, typ):
         optional = False
@@ -1215,7 +1219,6 @@ class QAPISchema(object):
         self._def_entity(QAPISchemaObjectType(name, info, base,
                                               self._make_members(data),
                                               None))
-        self._make_array_type(name)     # TODO really needed?
 
     def _make_variant(self, case, typ):
         return QAPISchemaObjectTypeVariant(case, typ)
@@ -1251,7 +1254,6 @@ class QAPISchema(object):
                                  QAPISchemaObjectTypeVariants(tag_name,
                                                               tag_enum,
                                                               variants)))
-        self._make_array_type(name)     # TODO really needed?
 
     def _def_alternate_type(self, expr, info):
         name = expr['alternate']
@@ -1264,7 +1266,6 @@ class QAPISchema(object):
                                     QAPISchemaObjectTypeVariants(None,
                                                                  tag_enum,
                                                                  variants)))
-        self._make_array_type(name)     # TODO really needed?
 
     def _def_command(self, expr, info):
         name = expr['command']
diff --git a/tests/qapi-schema/qapi-schema-test.json 
b/tests/qapi-schema/qapi-schema-test.json
index b48c6bd..4e2d7c2 100644
--- a/tests/qapi-schema/qapi-schema-test.json
+++ b/tests/qapi-schema/qapi-schema-test.json
@@ -40,6 +40,10 @@
   'data': { 'string0': 'str',
             'dict1': 'UserDefTwoDict' } }
 
+# dummy struct to force generation of array types not otherwise mentioned
+{ 'struct': 'ForceArrays',
+  'data': { 'unused1':['UserDefOne'], 'unused2':['UserDefTwo'] } }
+
 # for testing unions
 # Among other things, test that a name collision between branches does
 # not cause any problems (since only one branch can be in use at a time),
diff --git a/tests/qapi-schema/qapi-schema-test.out 
b/tests/qapi-schema/qapi-schema-test.out
index 93f6250..a6c80e0 100644
--- a/tests/qapi-schema/qapi-schema-test.out
+++ b/tests/qapi-schema/qapi-schema-test.out
@@ -86,6 +86,9 @@ object EventStructOne
     member struct1: UserDefOne optional=False
     member string: str optional=False
     member enum2: EnumOne optional=True
+object ForceArrays
+    member unused1: UserDefOneList optional=False
+    member unused2: UserDefTwoList optional=False
 enum MyEnum []
 object NestedEnumsOne
     member enum1: EnumOne optional=False
-- 
2.4.3