[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Feature request: --tag-json
From: |
Charles Howes |
Subject: |
Feature request: --tag-json |
Date: |
Thu, 17 Dec 2020 13:24:05 -0800 |
User-agent: |
Roundcube Webmail/1.3.10 |
Hi Ole;
Feature request: --tag-json
I'm dumping the state of my AWS resources using the CLI and getting JSON
output, which I wrangle into a single JSON file per resource type, but
it's an awful solution.
For example, given a list of profiles and regions, I might do this:
-------8<------cut here------8<---------
parallel --results "results/{1}/{2}" \
aws --profile {1} elb describe-load-balancers --region {2} \
:::: profiles.txt :::: regions.txt > /dev/null
export JQMERGE='(
input_filename|sub("/stdout$";"")|sub("^[^/]*/";"")|gsub("/";" ") ) as
$name | {"Name":$name,"Output":.}'
find results -name stdout -size +0 \
| parallel jq -r '"$JQMERGE"' \
| jq -s . > describe-load-balancers.json
------->8------cut here------>8---------
This has evolved; initially, I tried doing tricks with --tag-string to
generate json-style wrappers around the output, but it was a flimsy
hack.
Could you add a feature that would emit one json object per job to
stdout and result files, of the form:
{"Input":["arg1","arg2"],"Output":"..blah...blah...","Stderr":"...err...err..."}
I really can't think of a more robust way to do this, as any solution
would have to deduce/reconstruct the information that Parallel already
has. Here's a working example of what I mean:
-------8<------cut here------8<---------
parallel '
echo {1} {2} > stdout.{#} 2> stderr.{#} ;
jq -R -s . stdout.{#} > stdout.{#}.json;
jq -R -s . stderr.{#} > stderr.{#}.json;
jq -c -n --arg in1 "{1}" --arg in2 "{2}" \
--slurpfile stdout stdout.{#}.json \
--slurpfile stderr stderr.{#}.json \
'"'"'{Input:[$in1,$in2],Stdout:$stdout[0],Stderr:$stderr[0]}'"'"'
' ::: red green ::: cat
{"Input":["red","cat"],"Stdout":"red cat\n","Stderr":""}
{"Input":["green","cat"],"Stdout":"green cat\n","Stderr":""}
------->8------cut here------>8---------
I'd really like it if that could be replaced by this:
-------8<------cut here------8<---------
parallell --tag-json echo {1} {2} ::: red green ::: cat > output.json
------->8------cut here------>8---------
Thanks!
--
Penelope Fudd (parallel@ch.pkts.ca)
- Feature request: --tag-json,
Charles Howes <=