Re: Too much memory being used with

[Yves]

Nelson A. de Oliveira <naoliv <at> gmail.com> writes:

> 
> On Thu, Jul 24, 2014 at 2:05 PM, Martin Møller Skarbiniks Pedersen
> <traxplayer <at> gmail.com> wrote:
> > try remove the -j +0 parameter.
> 
> If I am not wrong, "-j +0" should be the same thing as no -j, right?
> But just in case, testing without -j the problem is exactly the same:
> a large amount of memory usage and no processing at all.
> 
> 


As (somewhat) counter-intuitive as this may seem, you have to drop the --eta 
or replace it with --progress. Per the man page --eta will force parallel to 
pre-read all the jobs before doing any real work, and that's what's eating up 
your RAM. Hard-learned lesson on my part as well, I'm wondering if a possible 
improvement wouldn't be to output a warning if/when processing is delayed 
beyond an excessive amount of time and/or (more importantly from my 
perspective) if RAM usage becomes unacceptable due to the pre-read implied by 
--eta.

-y