Re: --nest for nested argument files

[Ole Tange]

On Wed, May 4, 2011 at 6:11 PM, Hans Schou <chlor@schou.dk> wrote:
> On Wed, 4 May 2011, Ole Tange wrote:
>> On Wed, May 4, 2011 at 4:21 PM, Ole Tange <tange@gnu.org> wrote:
>> Hans' example with wav files could then be written as:
>>
>> parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: wavfiles
>> parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 ::: *.wav
>> ls *.wav | parallel oggenc -q {1} {2} {2.}_{1}.ogg ::: 1 5 10 :::: -
>
> Looks good. So, all these does the same:
>
> ls *.wav > list
>
> parallel echo {1} {2} ::: 1 2 3 :::: list
> parallel echo {1} {2} ::: $(seq 3) ::: *.wav
> parallel echo {1} {2} :::: <(seq 3) list
> parallel -a <(seq 3) echo {1} {2} ::: *.wav
> parallel -a <(seq 3) -a list echo {1} {2}
> parallel -a <(seq 3) -a <(ls *.wav) echo {1} {2}
> ls *.wav | parallel -a <(seq 3) echo {1} {2} :::: -

Yes.

> Is it really necessary to have the last ':::: -' ?
> Will this not do?
>  ls *.wav | parallel -a <(seq 3) echo {2} {1}

The problem is how parallel should know whether to look for stuff on stdin.

Currently if you do -a the parallel does not wait for input on stdin.
If you do neither ::: :::: nor -a then parallel knows it should start
reading from stdin.

That is why if you give -a then you need explicitly tell parallel to
read from stdin. -a - or :::: - will tell parallel that it should read
from stdin, too.

>>> The problem with these approaches is that if one of the files is
>>> called ':::' or '::::' then this will no longer work because the :::
>>> or :::: will confuse GNU Parallel:
>
> It is not very often I give my files that name.

And if you _know_ you have files called that, you can use --arg-sep
and --arg-file-sep to replace ::: and :::: with another string.


/Ole