-X doesn't quite work as documented or perhaps expected

[Jay Hacker]

Hello,

I'm using the new version 20110622.  I'd like to run one job, with all
arguments appended, and context replace.  I thought that's what -X
would do, from the manual:

-X
    Multiple arguments with context replace. Insert as many arguments
as the command line length permits.

However, what parallel actually seems to do is run one job for each
CPU core, and try to spread the arguments evenly.  On a machine with
four cores:

$ seq 1 5 | parallel -X echo file{}.jpg
file1.jpg file2.jpg
file3.jpg file4.jpg
file5.jpg

$ seq 1 5 | parallel -j2 -X echo file{}.jpg
file1.jpg file2.jpg file3.jpg
file4.jpg file5.jpg

This is arguably the right thing to do for a parallel processing tool,
but if that's the intent it would be nice to have it documented.  -m
and --xargs work similarly; however, this seems misleading if you're
trying to reproduce xargs' behavior exactly.

It also seems like parallel is trying to even out the load (number of
args per job), even if that means not using all cores; again, a
reasonable strategy, but somewhat surprising if you ask for -j4 and
only get 3 jobs.  That would also be nice to find in the docs.

A few other surprises: The manual says -L and -N "implies -X unless -m
is set," but they do not do context replace as -X does:

$ seq 1 5 | parallel -L10 echo file{}.jpg
file1 2 3 4 5.jpg

$ seq 1 5 | parallel -N10 echo file{}.jpg
file1 2 3 4 5.jpg

It would be nice to have context replace with -L and -N.  Given these,
I would expect -s to operate similarly, but I'm not sure what -s is
doing:

$ seq 1 5 | parallel -s100 echo file{}.jpg
file1.jpg
file2.jpg
file3.jpg
file4.jpg
file5.jpg

I can't seem to get -s to put more than one argument on a command no
matter what:

$ seq 1 5 | parallel -j1 -s10000 echo
1
2
3
4
5

And as a symptom of this, -x seems to be the default; that is, -s will
always exit if the limit is exceeded:

$ seq 1 5 | parallel -s1 echo
Command line too long (6 >= 1) at number 1: 1...

Finally, if you forget the argument to -s or -N or -L, it would be
nice to get a message indicating this, instead of crashing Perl:

$ seq 1 5 | parallel -s echo
Can't exec "-s": No such file or directory at
/usr/lib/perl5/5.8.8/IPC/Open3.pm line 246.
open3: exec of -s echo 1 failed at /usr/bin/parallel line 3182

So, I think I can solve my original problem with -Xj1, but seem to
have discovered a few others along the way.  Let me know if any of
these should go in a bug report.