bug#40693: 28.0.50; json-encode-alist changes alist

[Basil L. Contovounesios]

tags 40693 + patch
quit

0001-Various-json.el-improvements.patch
Description: Text Data

"Basil L. Contovounesios" <contovob@tcd.ie> writes:

> I have a local WIP branch where I am refactoring parts of json.el to
> improve performance, and avoiding the proposed copy-sequence is one of
> the fixes.

Large patch attached (fear not (too much); the bulk of it is new
regression tests).  CCing João because it also touches jsonrpc.el.

The only backward-incompatible part of it should be the stricter number
parsing.  Until now[1], json.el accepted numbers with a leading plus
sign, leading zeros, and no integer component, all of which are
forbidden by the JSON spec[2] and rejected by Emacs' native JSON parsing
functions.  I was initially willing to ignore this according to "be
liberal in what you accept and strict in what you output," but given
that libraries are already treating json.el and json.c functions as
interchangeable, I think json.el should be fixed.  Who knows, it may
even help to catch some bad JSON generation somewhere, somewhen.

[1]: (json-read-number): New arg.  Handle explicitly signed numbers.
7712319db8 2008-08-28 20:19:17 +0000
https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=7712319db8db97391059925b27ae71f304eee7d2

[2]: https://tools.ietf.org/html/rfc8259#section-6

> I've also found a couple of test JSON files to benchmark against.

I downloaded the three large (sizes 617KB-2.2MB) JSON files here:

https://github.com/miloyip/nativejson-benchmark/tree/master/data

and ran the following benchmark:

json-bench.el
Description: application/emacs-lisp

like so:

$ emacs -Q -batch -f batch-byte-compile json-bench.el
$ for e in /old/emacs /new/emacs; do "$e" -Q -script json-bench.elc *.json; done

with the following (manipulated, for easier comparison) output:

decode canada.json
old
(2.147126599 64 0.714606792)
(2.184767529 64 0.772983248)
(2.190970555 64 0.7805527100000003)
(2.1976262359999996 64 0.786285103)
new
(2.240072109 62 0.701617291)
(2.2702309019999998 62 0.751942913)
(2.270719533 62 0.753646107)
(2.2844341569999997 62 0.7555721750000002)

This file is 2MB of numbers, so I'm speculating that the slight slowdown
is due to the stricter number regexp.

encode canada.json
old
(5.672111511 172 1.9869896950000001)
(5.6662210150000005 172 1.9884779629999993)
(5.670608892 172 1.9894344889999998)
(5.670605633999999 172 1.9883765339999986)
new
(2.4715460570000003 96 1.1121157440000005)
(2.4745830389999997 96 1.1193736730000001)
(2.465779169 96 1.1147967909999998)
(2.4715383390000003 96 1.1174477880000007)

decode citm_catalog.json
old
(1.010838497 34 0.3915659290000004)
(1.006386333 34 0.390746824999999)
(1.06575967 34 0.4085684829999998)
(1.0045083860000001 34 0.3905320579999998)
new
(0.9698266640000001 34 0.39840883500000057)
(0.969682733 34 0.3984373899999998)
(0.972970641 34 0.4009393059999997)
(0.974868517 34 0.40203729300000113)

encode citm_catalog.json
old
(5.063678788 292 3.2036687829999995)
(5.056699193 292 3.201438146000001)
(5.048888015999999 292 3.1970336089999982)
(5.046339092 292 3.197215603)
new
(4.38071611 268 3.0482562499999997)
(4.327264702 268 3.011219348999999)
(4.320279397999999 268 3.0082109699999986)
(4.322212972 268 3.010901667999999)

decode twitter.json
old
(0.788541053 32 0.34238300700000224)
(0.776468868 32 0.33285421699999773)
(0.777424967 32 0.3338445579999991)
(0.779168795 32 0.3343039160000032)
new
(0.7064887900000001 32 0.34372805699999986)
(0.6965333029999999 32 0.33550817600000116)
(0.700879238 32 0.33781746400000046)
(0.701126792 32 0.3365076699999996)

encode twitter.json
old
(2.640938486 163 1.6357819669999998)
(2.64012542 163 1.6363964359999983)
(2.6377853709999997 163 1.6349542369999988)
(2.639091697 163 1.635746981000004)
new
(2.2311199000000004 144 1.5285102250000016)
(2.227288643 144 1.5259519570000002)
(2.229322183 144 1.5265554599999973)
(2.223512976 144 1.5231076530000003)

WDYT?  Thanks,

-- 
Basil