Re: [dmidecode] Question: YAML and/or JSON output support

[Jean Delvare]

Hi Matthew,

Sorry for the late answer.

On Sat, 11 Jan 2020 10:29:16 -0800, Matthew Giassa wrote:
> Good day,
> 
> I'm working on a hobby project involving various dev boards, and I use
> `dmidecode` to help distinguish what "type" of board the software is
> running on, at run time. Some of the programs that run on the various
> boards are written in different languages, including C, C++, Golang,
> Python 2/3, and so on.
> 
> While there are dmidecode-specific parsers of varying levels of maturity
> for most of these languages, it would be preferable if `dmidecode`
> itself could directly export to YAML and/or JSON formats (no need to use
> various parsers with differing licenses).

What kind of information do you need? The most valuable "keys" to
identify a system are available through the command line parameter
"--string". This can be called from any language and should limit the
parsing effort to a bare minimum.

For the more general discussion about improving the output of
dmidecode, you are not the first one to ask for it. The most recent
request was from Erwan Velu (Cc'd):

https://lists.nongnu.org/archive/html/dmidecode-devel/2018-09/msg00018.html

There has been at least one more public request for XML output before
that (that I can no longer get my hands on, sorry), and the first trace
in my mail box dates back to 2005 in a private thread which sadly died
with no result.

Unfortunately this is easier said than done, see below.

> Would this feature be allowable, for example:
> * As a direct modification to `dmidecode` itself (i.e. additional
>   feature; requires linking against additional libraries).

Yes, linking against additional libraries would be acceptable, ideally
as an option. Generating a machine-readable output may not require a
library though, as opposed to parsing it.

> * As a variant of `dmidecode`: i.e. wrap the feature in #ifdef's, and
>   have the build/makefile script compile multiple binaries, including a
>   "stock" `dmidecode`, and a second binary, `dmidecode_json` (only the
>   latter would link against additional external libraries). Additional
>   libraries are needed for the `dmidecode` package itself, but not the
>   "stock" (i.e. no JSON/YAML capability) `dmidecode` binary. Could
>   possibly result in two separate downstream packages derived from the
>   same source.

I don't like the idea of building multiple binaries. It is a pain for
distributions to package and results in a lot of integration headaches
in my experience. I think it is much easier to have one binary with
always the same name. Distributions can enable specific features
depending on their needs and what libraries are available - they do
that for a lot of packages already.

> * Just fork/branch the source: I could also just fork the Github-mirror
>   repo if this functionality isn't wanted/needed in the mainline repo. I
>   imagine others would benefit from this feature, based on the existence
>   of multiple dmidecode-specific parsers, so this route only makes sense
>   if this feature doesn't belong in `dmidecode` mainline/master.

Forking is the worst option and should be avoided at pretty much any
cost.

The conclusion you draw based on the existence of dmidecode output
parsers is probably wrong, I think people simply either did not even
bother asking, or they lost patience because nothing was happening on
the dmidecode front and they needed it yesterday. That does not mean it
should not be done.

> What do the developers/maintainers think of these possible approaches to
> implementing such a feature?

I'd be happy to have this feature added to dmidecode mainline, really.
The problem is that the amount of work needed to get there is huge and
I simply never can find the time to jump into something that big.

Specifically there are 3 hurdles to jump over:

1* The dmidecode code was never designed for that. Currently
   *everything* that is displayed when you call dmidecode comes from
   open-coded printfs. There are no intermediate structures nor
   temporary buffers, everything is printed piece by piece as it is
   parsed from the DMI table. In many cases a single line is actually
   the result of 3 printfs. This is diametrically opposed to what you
   need to do to allow for alternative output formats. So before
   anything else, dmidecode need to be entirely rewritten. 5200 lines
   of code. I'm not even sure where to start, this is such a massive
   design change really.

2* Machine-readable is a generic term, we would need to settle on an
  actual format. This appears to be a moving target, over time people
  have been asking for SGML (I think), XML, JSON and now YAML (never
  heard of that one before). Maybe a few more that I do not remember.
  Depends on the trend of the moment. I don't really want to support 4
  different formats, although we certainly want to design the new code
  in such a way that using a different format later wouldn't be too much
  work.

3* Even after deciding for a format, it says nothing about what the
   output should look like. What should be the structure of the output?
   What should be a key, what should be a value? How do you order the
   entries? How to you deal with specification changes such as new DMI
   structures, new entries added to older structures, renamed fields
   (yes it did happen before), obsoleted fields? As usual, devil is in
   the details. Making such format decisions that we then need to stick
   to pretty much forever is a huge deal, at least for me.

I may give a stab to 1* during the upcoming hack week at SUSE in
February. However I do not feel qualified at all for 2* and 3*, as I
won't be using the machine-readable output myself. So it would be much
better if people who need it would actually define what the output
format should be - down to the details. One of the reasons why
dmidecode does not have a machine-readable output yet is because that
never happened.

I hope this clarifies,
-- 
Jean Delvare
SUSE L3 Support