Re: Does Libvirt's json parser support single quoted string in qmp json string?

[John Snow]

On 2/3/20 11:27 AM, Eric Blake wrote:
> [adding Markus]
> 
> On 2/3/20 4:13 AM, Daniel P. Berrangé wrote:
>> On Fri, Jan 31, 2020 at 06:44:42AM -0600, Eric Blake wrote:
>>> On 1/31/20 4:38 AM, Peter Luo wrote:
>>>
>>>> error: internal error: cannot parse json {"execute": "block-commit",
>>>> "arguments": { "device": "drive-virtio-disk2", "job-id": "job100",
>>>> "base":'json:{"encrypt.key-secret":"vol-38973xjl.secret","driver":"qcow2","file":{"driver":"file","filename":"/pitrix/data/container/vol-38973xjl.img"}}',
>>>> "top": "/pitrix/data/container/vol-38973xjl_ss-2tw7v0mm.img"}}:
>>>> lexical error: invalid char in json text.
>>>>
>>>>             , "job-id": "job100",
>>>> "base":'json:{"encrypt.key-secret":"vo
>>>>
>>>>                        (right here) ------^
>>>>
>>>
>>> qemu's QMP language has an extension where it accepts 'string' in
>>> addition
>>> to the JSON "string".  But it uses a hand-rolled parser, so it can do
>>> whatever it wants.
>>
>> Can we deprecate & remove this extension in QEMU ?
> 
> We could start a deprecation clock, if desired, but I don't know how
> many external users would be impacted (at least qemu's testsuite heavily
> relies on the extension of single quotes).
> 
> Are there any third-party libraries that parse JSON5? https://json5.org/
> documents that one of the nice features of JSON5 is that single quotes
> work out of the box.  Right now, even though qemu does NOT parse the
> same thing as pure JSON (which is
> https://datatracker.ietf.org/doc/rfc8259/), it does appear to parse a
> subset of JSON5, where the additional features of JSON5 (such as
> allowing a trailing comma, escaped newlines within strings, allowing
> bareword key:"value" syntax, a larger set of accepted numeric values,
> allowing comments) don't seem that onerous.
> 
>>
>> If we're going to call QMP protocol JSON, then IMHO QEMU should follow
>> the JSON spec as closely as possible, without extensions.
> 
> As it is, qemu explicitly does NOT parse all valid JSON - it rejects
> non-ASCII bytes (whether as UTF-8 or as \u escape sequences) in strings.
>  So the fact that it already extensions in some places and limitations
> in others is a burden for clients to be aware of.
> 
>>
>> On the flip side, if we're going to support extensions like single
>> quoting,
>> then we should make it clear to applications that this is not really JSON
>> and that they need to provide an impl that is 100% matching QEMU's
>> dialect.
>> This effectively means they need just import a copy of QEMU's code.
>>
>> Regards,
>> Daniel
>>
> 

More anecdata: qmp-shell will also accept either strict JSON or what it
calls "fuzzy" JSON, which allows single quotes, Python spellings for
boolean constants, and other deviations.

However, when using pretty-print, it will always show you the *strict*
json form of your input.

If we are confusing users, that's bad; but there's some use for the
single quotes in our code base. Following a standard would be nice.

JSON5 might be "niche", but so what? If we happen to more closely adhere
to JSON5 than vanilla JSON, we could still make sure we adhere to it and
we get to say that we accept "strict JSON5." This is at least unambiguous.

I'm more inclined to head towards JSON5 instead of heading back to
strict JSON.

--js