I am trying to concatenate a set of files to another set of files. To be more specific I want to parallelize the following loop:
for ((i=0,i<=99,i+=4));do
cat ${filelist1[@]:$i:4} ${filelist2[@]:$i:4} > ${filelist3[@]:$i}
done
So I decided to generate the list of files:
for ((i=0,i<=99,i+=4));do
f1in=("${f1in[@]}" "`echo ${filelist1[@]:$i:4}`")
f2in=("${f2in[@]}" "`echo ${filelist2[@]:$i:4}`")
fout=("${fout[@]}" "${filelist3[@]:$i}")
done
and pass these as arguments to parallel:
parallel --xapply cat {1} {2} ">" {3} ::: "f1in[@]" ::: "f2in[@]" ::: "fout[@]"
However the above generates an error since the first element of f1in contains blanks that are consider as characters by cat and as a result cat cannot find the files `echo ${filelist1[@]:$i:4}`. On the contrary a simple cat works as expected:
cat f1in[0] f2in[0] > fout[0]
There is something I'm missing here. Could you please help me or provide a workaround for what I'm trying to do? Is this a valid use of parallel?
Thank you in advance.
Kostas