[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Light weight support for JSON

From: Robert E. Griffith
Subject: Re: Light weight support for JSON
Date: Thu, 1 Sep 2022 00:10:15 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.11.0

I am on vacation and just skimmed this long thread so I might have missed some of the context but I wanted throw out that I recently wrote a loadable plugin that among other things, can convert JSON to and from bash arrays.

Anyone interested can check it out at these git repos. https://github.com/junga-com/bg-core and https://github.com/junga-com/bg-core-bash-builtins The bg-core project is an all bash implementation. If the compiled bgCore bash loadable from bg-core-bash-builtins is available it runs much faster but the functionality is the same either way. (BTW, the bg-core-bash-builtins is a C project which includes a module bg_bashAPI.h/c that is a wrapper intended to make it easier to convert bash code to C -- i.e. create and manipulate shell variables)

The bg-core library introduces the concept of Classes and Objects to bash. A key feature to enable objects is a concept of 'allocating' bash variables on a 'heap'. The 'heap' just means that heap variables use a global variable name schema with a random component so that code can create them dynamically without conflict.  When an associative array element contains a string that matches the pattern of a heap variable name (e.g. heap_<attributes>_<randomPart>) or an object reference (_bgclassCall <classname> <heapVarName>), the object syntax and other aware functions like the toJSON and fromJSON function recognizes it as an heap variable reference which can be an associative array. The upshot is that you can have nested arrays (elements of arrays can themselves be arrays).

With the ability to have nested arrays, its very easy to map any arbitrary JSON data to bash and vica versa.  JSON {} objects map to bash associative arrays and JSON [] lists/arrays map to bash numeric arrays. Each time a {} or [] construct is encountered reading JSON data, a new bash array is allocated on the 'heap (i.e. a new random variable name is created and used to create a global var) and its name is stored in the associative array element of its parent.

I have been using this for a while now and its all working quite nicely.


On 8/31/22 11:11, Chet Ramey wrote:
On 8/29/22 2:03 PM, tetsujin@scope-eye.net wrote:
On 2022-08-29 11:43, Chet Ramey wrote:
On 8/28/22 2:11 PM, Lawrence Velázquez wrote:
On Sun, Aug 28, 2022, at 9:24 AM, Yair Lenga wrote:
Wanted to get feedback about the following "extensions" to bash that will
make it easier to work with simple JSON object.  (...)
just to make it easier to build bash scripts that cover the very common use case of submitting REST requests with curl (checking results, etc), and to
perform simple processing of JSON files.

I do not think bash needs to sprout functionality to support every
data-exchange format of the month.

Loadable builtins are the way to do this

In that case could I suggest providing a simplified, stable programming interface for loadable builtins?  (I understand this is not a trivial thing, of course.)

I'm not opposed, but it's not going to be a high priority for me. I have
more pressing things to do. That is not to minimize this goal; I simply
put my time into things that impact the maximum number of users.

If someone wanted to make a pass at defining a useful subset of the (quite
large) internal bash API, that would be a start.

As I understand it (having made a few loadable built-ins myself, but possibly a bit ignorant/hazy on the details) - the process involves a fair bit of hands-on with the nitty-gritty of the version of Bash you're building against. A loadable needs to deal directly with implementation details of how Bash handles variables, open file handles, etc., and resulting code (source and binary) is very sensitive
to changes in Bash.

I don't know. I suppose it depends on the complexity of what you're trying
to do. I've not had to update the source in the examples when moving to a
new bash version very often. It's always a good idea to rebuild a loadable
against the bash version you're going to load it into, though.

For me personally I was struggling with questions like, how do I make a loadable module that sets a variable? Sounds simple but it seems to require a fair bit of attention to implementation details of the shell to get it right. What if it's a
local variable? What if it's an array or hash element? And so on.

There are individual functions to do all of these things, but it does
require knowing about them, no doubt.

It would also help
greatly if the shell could internally handle hierarchical data in

That's a fundamental change. There would have to be a better reason to make
it than handling JSON.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]