Re: cygwin xargs limitation: ARG_MAX depends on command

[Bob Proulx]

Eric Blake wrote:
> But even if cygwin 1.5.19 were to define ARG_MAX and changes
> sysconf(_SC_ARG_MAX) to 32k instead of 1 meg, this would unfairly
> penalize cygexec mounts, which can handle much bigger command lines.

I know this sounds like it would unfairly limit all uses because some
uses are smallish at 32k.  But is this *really* a hardship?  That is
still a pretty good size arg list.  I think we are probably getting up
into the very small percentage points of performance improvement here
with buffer sizes larger than that.

If it is not too much trouble could you do post some simple benchmarks
showing that larger buffer sizes are significantly better than the 32k
buffer sizes?  If so then it would help to promote your case.

For example on my Debian Sarge amd64 system running 4.1.20, and trying
to be a little more compact for the mailing list:

  (export TIMEFORMAT='real %3lR  user %3lU  sys %3lS'
  for i in 1024 2048 4096 8192 16384 20480; do
    printf "timing $i: "
    yes | head --bytes=10m | bash -c "time xargs -s$i /bin/echo > /dev/null"
  done)

  timing 1024: real 0m5.693s  user 0m2.091s  sys 0m3.343s
  timing 2048: real 0m3.358s  user 0m1.406s  sys 0m1.752s
  timing 4096: real 0m3.400s  user 0m1.344s  sys 0m1.830s
  timing 8192: real 0m3.483s  user 0m1.375s  sys 0m1.908s
  timing 16384: real 0m3.354s  user 0m1.379s  sys 0m1.774s
  timing 20480: real 0m3.466s  user 0m1.329s  sys 0m1.895s

Those numbers are just so close together.  I could increase the data
set size.  But let's try this on a k6 450MHz which has more spread.

  timing 1024: real 0m49.509s  user 0m21.870s  sys 0m25.310s
  timing 2048: real 0m30.735s  user 0m14.130s  sys 0m13.960s
  timing 4096: real 0m31.688s  user 0m14.040s  sys 0m15.850s
  timing 8192: real 0m30.394s  user 0m13.970s  sys 0m14.080s
  timing 16384: real 0m30.547s  user 0m14.480s  sys 0m13.890s
  timing 20480: real 0m32.414s  user 0m14.260s  sys 0m15.940s

It just very quickly does not matter.  Let me guess, without data and
perhaps wrongly, that this will not be a big performance difference on
cygwin either.  So probably on cygwin I would go ahead and make that
size restriction so that it works correctly all of the time.  But I am
not mothering over that platform and I know that those that do will
want to do better so personally I sympathize.  But it just does not
seem to me like a performance lever.

> Is there any other system where the max argument length depends on
> the command?

It does seem like a very unique situation specific to that platform.

Bob