Use R to manage results from GNU Parallel

[Ole Tange]

If you are not using R: Read no further.

GNU Parallel can save results into a dir structure:

$ parallel --results my/res/dir --header : 'echo {};seq {1}' :::
myvar1 1 2 ::: myvar2 A B
$ find my/ -type f
my/res/dir/myvar1/1/myvar2/B/stderr
my/res/dir/myvar1/1/myvar2/B/stdout
my/res/dir/myvar1/1/myvar2/A/stderr
my/res/dir/myvar1/1/myvar2/A/stdout
my/res/dir/myvar1/2/myvar2/B/stderr
my/res/dir/myvar1/2/myvar2/B/stdout
my/res/dir/myvar1/2/myvar2/A/stderr
my/res/dir/myvar1/2/myvar2/A/stdout

I have now written an R function that reads these files and put them
into a matrix:

> load_parallel_results("my/res/dir")
      myvar1 myvar2 stdout      stderr
 [1,] "1"    "A"    "1 A\n1"    ""
 [2,] "1"    "B"    "1 B\n1"    ""
 [3,] "2"    "A"    "2 A\n1\n2" ""
 [4,] "2"    "B"    "2 B\n1\n2" ""

If you have loads of results then you probably want to be able to some
analysis on those, and R is a reasonable tool for that.

The function to read the results is in
http://git.savannah.gnu.org/cgit/parallel.git/plain/src/load-parallel-results.r
or in:

  git clone git://git.savannah.gnu.org/parallel.git

It would be good if you could test it out, and if you could improve my
feeble R skills (you will see the code is not very elegant).


/Ole