Re: Saving output

[Ole Tange]

On Wed, Dec 1, 2010 at 2:46 PM, Mike Rochford <mike@mikerochford.com> wrote:
> I use four machines to run a command and record output on 10k+ hosts
> using GNU parallel. I am having trouble saving the output to a text
> file. Ultimatly I would like to see one log file for each machine I am
> using to issue the command. Is there a easy way to do this?
>
> Below is what I am currently doing:
>
> cat hostlist | parallel -Shost1,host2,host3,: -j 15 --eta "ssh -i
> <key> root@{}  "uname -a" > /tmp/serveroutput/{}"

I would do not use 4 hosts to do the ssh from but do it from a single host:

cat hostlist | parallel -j60 --eta ssh -i keyfile root@{} uname -a
">"/tmp/serveroutput/{}

What do you benefit from using 4 hosts? I would think it is a bit
messy having the outputs spread on 4 hosts.

> once this is complete I cat ever file in that directory into a single file.

If you do this anyway you might aswell just do this (remember output
is not getting mixed as --group is default):

cat hostlist | parallel -j60 --eta ssh -i keyfile root@{} uname -a
>/tmp/all_outouts

And if you use GNU Parallel 20101202 then --eta will not be sent to
standard output and thus not mess up your all_outputs-file.

When I do stuff like the above I often want to see which command is
being run in the logfile, so I add -v. Or I may want the sequence to
remain the same, so I add -k.

cat hostlist | parallel -j60 -vk --eta ssh -i keyfile root@{} uname -a
>/tmp/all_outouts


/Ole