[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Light weight support for JSON

From: Yair Lenga
Subject: Re: Light weight support for JSON
Date: Sun, 28 Aug 2022 17:50:13 -0400

First, thanks for taking the time to read and provide your thoughts. This
is the real value of the discussion/

Second: I'm NOT trying to argue that there isn't valid use for
combining bash/curl/jq, Nor do I suggest adding JSON as first class object
to bash (Python/node/Perl/Groovy are way ahead  ...).

I hope to get feedback from other readers for the news group that may find
this approach useful.

I'll take the very common use case of using AWS CLI, which does produces
JSON response for most calls. Processing the response, while possible with
JQ, it is challenging to many junior (and intermediate) developers. In many
cases, they fall into the traps that I mentioned above - performance
(excessive forking or fork/exec), or code that is hard to read (I've seen
really bad code - combining pipes of JQ/awk/sed). I'm trying to address
those cases. Almost always they fail to properly handle objects with while
space, new-lines, etc.

To be practical, I'll try to follow the loadable extension path, and see
how much I can get thru that path. Possibly will make sense to continue
discussion with a concrete implementation. I believe the necessary commands

json_read -a data-array -m meta-array -r root-obj
Parse the stdin input and into data-array with the items (as described
above), and the meta-array with helper information (length, prop list under
each node) - to help iterating.

json_write -v variable -a data-array -m meta-array -r root
The reverse - generate the JSON for associated array following '.' naming

json_add -a data-array [-m meta-array] [-r root] key1=value1 key2=value2
Helper to add items into associative array representing JSON object,
Autodetect type, with the ability to force stringfication using format

On Sun, Aug 28, 2022 at 3:22 PM John Passaro <john.a.passaro@gmail.com>

> interfacing with an external tool absolutely seems like the correct answer
> to me. a fact worth mentioning to back that up is that `jq` exists. billed
> as a sed/awk for json, it fills all the functions you'd expect such an
> external tool to have and many many more. interfacing from curl to jq to
> bash is something i do on a near daily basis.
> https://stedolan.github.io/jq/
> On Sun, Aug 28, 2022, 09:25 Yair Lenga <yair.lenga@gmail.com> wrote:
>> Hi,
>> Over the last few years, JSON data becomes a integral part of processing.
>> In many cases, I find myself having to automate tasks that require
>> inspection of JSON response, and in few cases, construction of JSON. So
>> far, I've taken one of two approaches:
>> * For simple parsing, using 'jq' to extract elements of the JSON
>> * For more complex tasks, switching to python or Javascript.
>> Wanted to get feedback about the following "extensions" to bash that will
>> make it easier to work with simple JSON object. To emphasize, the goal is
>> NOT to "compete" with Python/Javascript (and other full scale language) -
>> just to make it easier to build bash scripts that cover the very common
>> use
>> case of submitting REST requests with curl (checking results, etc), and to
>> perform simple processing of JSON files.
>> Proposal:
>> * Minimal - Lightweight "json parser" that will convert JSON files to bash
>> associative array (see below)
>> * Convert bash associative array to JSON
>> To the extent possible, prefer to borrow from jsonpath syntax.
>> Parsing JSON into an associative array.
>> Consider the following, showing all possible JSON values (boolean, number,
>> string, object and array).
>> {
>>     "b": false,
>>     "n": 10.2,
>>     "s: "foobar",
>>      x: null,
>>     "o" : { "n": 10.2,  "s: "xyz" },
>>      "a": [
>>          { "n": 10.2,  "s: "abc", x: false },
>>          {  "n": 10.2,  "s": "def" x: true},
>>      ],
>> }
>> This should be converted into the following array:
>> -------------------------------------
>> # Top level
>> [_length] = 6                            # Number of keys in object/array
>> [_keys] = b n s x o a            # Direct keys
>> [b] = false
>> [n] = 10.2
>> [s] = foobar
>> [x] = null
>> # This is object 'o'
>> [o._length] = 2
>> [o._keys] = n s
>> [o.n] = 10.2
>> [o.s] = xyz
>> # Array 'a'
>> [a._count] =  2                   # Number of elements in array
>> # Element a[0] (object)
>> [a.0._length] = 3
>> [a.0._keys] = n s x
>> [a.0.n] = 10.2
>> [a.0.s] = abc
>> [a.0_x] = false
>> -------------------------------------
>> I hope that example above is sufficient. There are few other items that
>> are
>> worth exploring - e.g., how to store the type (specifically, separate the
>> quoted strings vs value so that "5.2" is different than 5.2, and "null" is
>> different from null.
>> I will leave the second part to a different post, once I have some
>> feedback. I have some prototype that i've written in python - POC - that
>> make it possible to write things like
>> declare -a foo
>> curl http://www.api.com/weather/US/10013 | readjson foo
>> printf "temperature(F) : %.1f Wind(MPH)=%d" ${foo[temp_f]}, ${foo[wind]}
>> Yair

reply via email to

[Prev in Thread] Current Thread [Next in Thread]