Re: Bug in GNU Parallel 20200622

[Ole Tange]

On Fri, Aug 7, 2020 at 10:22 PM Mark Setchell <marksetchell@icloud.com> wrote:

Hi Mark

I was wondering when you would show up here :)

> I run the following command on macOS Catalina 10.15
>
> seq -w 270000 | parallel -X touch
>
> It reports the version number is 20200622, bugid: open3-GLOB(0x7fc72e7f7b98)

I can reproduce this on MacOS X.

> It seems to be mistaking the maximum number of characters in a command's 
> parameters with the maximum number of parameters - when they are clearly 
> different things.

That, however, is not what is happening.

What is happening is that something on MacOS X affects how long a
command line can be. I as far as I can tell it includes:

* number of exported variables/functions and names
* the size of the same
* how many arguments are passed to a command

and maybe other things.

So a command of length 260000 can be fine if there are only 2
arguments and few exported environment variables, but not if there are
100000 arguments adding up to the same length or if there are many
exported variables.

This does not happen on GNU/Linux. Initially I thought it might be a
problem with a version of Bash, but running the same version of Bash
on GNU/Linux does not show this issue.

Using '-s 27000' will work in all the cases I have found so far. But
to me that is a highly unsatisfactory workaround, and it is likely
that it will break in some situations. I would much rather understand
why this happens, and how to compute the maximal size.

If you find some logic in that, please submit a patch.

(https://unix.stackexchange.com/questions/604066/macosx-number-of-arguments-on-command-line
was created to solve part of the issue).


/Ole