Re: [Qemu-devel] [PATCH 0/9] Our QAPI parser is a hack, replace it

[Anthony Liguori]

Markus Armbruster <address@hidden> writes:

> If you think I'm exaggerating, check out the list of issues in PATCH
> 3/9.

You are not.

However, I think we can drop the whole thing and just use the JSON
module in Python.  The bit below seems to work:

  import json.decoder, re
  from ordereddict import OrderedDict
  
  WHITESPACE = re.compile(r'(#.*\n|[ \r\t\n]*)*', re.MULTILINE)
  
  def make_object(pairs):
      return OrderedDict(pairs)
  
  def qapi_parse(data):
      _w = WHITESPACE.match
      idx = 0
      while idx < len(data):
          idx = _w(data, idx).end()
          if idx == len(data):
              break
          decoder = json.decoder.JSONDecoder(object_pairs_hook=make_object)
          obj, idx = decoder.raw_decode(data, idx)
          yield obj
  
  if __name__ == '__main__':
      with open('qapi-schema.json', 'r') as fp:
          data = fp.read().replace("'", '"')
  
      exprs = list(qapi_parse(data))
      print exprs

Regards,

Anthony Liguori

>
> Markus Armbruster (9):
>   tests: QAPI schema parser tests
>   tests: Use qapi-schema-test.json as schema parser test
>   qapi.py: Restructure lexer and parser
>   qapi.py: Decent syntax error reporting
>   qapi.py: Reject invalid characters in schema file
>   qapi.py: Fix schema parser to check syntax systematically
>   qapi.py: Fix diagnosing non-objects at a schema's top-level
>   qapi.py: Rename expr_eval to expr in parse_schema()
>   qapi.py: Permit comments starting anywhere on the line
>
>  configure                                      |   2 +-
>  qapi-schema-test.json                          |  53 ------
>  scripts/qapi.py                                | 225 
> +++++++++++++++----------
>  tests/Makefile                                 |  28 ++-
>  tests/qapi-schema/empty.exit                   |   1 +
>  tests/qapi-schema/empty.out                    |   3 +
>  tests/qapi-schema/funny-char.err               |   1 +
>  tests/qapi-schema/funny-char.exit              |   1 +
>  tests/qapi-schema/funny-char.json              |   2 +
>  tests/qapi-schema/indented-expr.exit           |   1 +
>  tests/qapi-schema/indented-expr.json           |   2 +
>  tests/qapi-schema/indented-expr.out            |   3 +
>  tests/qapi-schema/missing-colon.err            |   1 +
>  tests/qapi-schema/missing-colon.exit           |   1 +
>  tests/qapi-schema/missing-colon.json           |   2 +
>  tests/qapi-schema/missing-comma.err            |   1 +
>  tests/qapi-schema/missing-comma.exit           |   1 +
>  tests/qapi-schema/missing-comma.json           |   2 +
>  tests/qapi-schema/non-objects.err              |   1 +
>  tests/qapi-schema/non-objects.exit             |   1 +
>  tests/qapi-schema/non-objects.json             |   2 +
>  tests/qapi-schema/qapi-schema-test.exit        |   1 +
>  tests/qapi-schema/qapi-schema-test.json        |  53 ++++++
>  tests/qapi-schema/qapi-schema-test.out         |  19 +++
>  tests/qapi-schema/quoted-structural-chars.err  |   1 +
>  tests/qapi-schema/quoted-structural-chars.exit |   1 +
>  tests/qapi-schema/quoted-structural-chars.json |   1 +
>  tests/qapi-schema/test-qapi.py                 |  27 +++
>  tests/qapi-schema/unclosed-object.err          |   1 +
>  tests/qapi-schema/unclosed-object.exit         |   1 +
>  tests/qapi-schema/unclosed-object.json         |   1 +
>  tests/qapi-schema/unclosed-string.err          |   1 +
>  tests/qapi-schema/unclosed-string.exit         |   1 +
>  tests/qapi-schema/unclosed-string.json         |   2 +
>  34 files changed, 298 insertions(+), 146 deletions(-)
>  delete mode 100644 qapi-schema-test.json
>  create mode 100644 tests/qapi-schema/empty.err
>  create mode 100644 tests/qapi-schema/empty.exit
>  create mode 100644 tests/qapi-schema/empty.json
>  create mode 100644 tests/qapi-schema/empty.out
>  create mode 100644 tests/qapi-schema/funny-char.err
>  create mode 100644 tests/qapi-schema/funny-char.exit
>  create mode 100644 tests/qapi-schema/funny-char.json
>  create mode 100644 tests/qapi-schema/funny-char.out
>  create mode 100644 tests/qapi-schema/indented-expr.err
>  create mode 100644 tests/qapi-schema/indented-expr.exit
>  create mode 100644 tests/qapi-schema/indented-expr.json
>  create mode 100644 tests/qapi-schema/indented-expr.out
>  create mode 100644 tests/qapi-schema/missing-colon.err
>  create mode 100644 tests/qapi-schema/missing-colon.exit
>  create mode 100644 tests/qapi-schema/missing-colon.json
>  create mode 100644 tests/qapi-schema/missing-colon.out
>  create mode 100644 tests/qapi-schema/missing-comma.err
>  create mode 100644 tests/qapi-schema/missing-comma.exit
>  create mode 100644 tests/qapi-schema/missing-comma.json
>  create mode 100644 tests/qapi-schema/missing-comma.out
>  create mode 100644 tests/qapi-schema/non-objects.err
>  create mode 100644 tests/qapi-schema/non-objects.exit
>  create mode 100644 tests/qapi-schema/non-objects.json
>  create mode 100644 tests/qapi-schema/non-objects.out
>  create mode 100644 tests/qapi-schema/qapi-schema-test.err
>  create mode 100644 tests/qapi-schema/qapi-schema-test.exit
>  create mode 100644 tests/qapi-schema/qapi-schema-test.json
>  create mode 100644 tests/qapi-schema/qapi-schema-test.out
>  create mode 100644 tests/qapi-schema/quoted-structural-chars.err
>  create mode 100644 tests/qapi-schema/quoted-structural-chars.exit
>  create mode 100644 tests/qapi-schema/quoted-structural-chars.json
>  create mode 100644 tests/qapi-schema/quoted-structural-chars.out
>  create mode 100644 tests/qapi-schema/test-qapi.py
>  create mode 100644 tests/qapi-schema/unclosed-object.err
>  create mode 100644 tests/qapi-schema/unclosed-object.exit
>  create mode 100644 tests/qapi-schema/unclosed-object.json
>  create mode 100644 tests/qapi-schema/unclosed-object.out
>  create mode 100644 tests/qapi-schema/unclosed-string.err
>  create mode 100644 tests/qapi-schema/unclosed-string.exit
>  create mode 100644 tests/qapi-schema/unclosed-string.json
>  create mode 100644 tests/qapi-schema/unclosed-string.out
>
> -- 
> 1.7.11.7