[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Problem with thousands of small jobs.
From: |
Ole Tange |
Subject: |
Re: Problem with thousands of small jobs. |
Date: |
Wed, 11 Feb 2015 09:02:07 +0100 |
On Fri, Feb 6, 2015 at 2:20 PM, xmoon 2000 <xmoon2000@googlemail.com> wrote:
> On my 32 core machine works OK, BUT there is a "lull" in processing
> every few seconds as new jobs are started, once the current crop have
> completed.
I have assumed that the disk is inactive and that GNU Parallel is
taking 100% of a core, and that you would like to use all your cores
to 100%. But maybe that assumption is wrong. Maybe GNU Parallel is
waiting for disk I/O? Maybe the system running low on RAM and swapping
(dataprocessing jobs often use the most memory just before they exit)?
Can you describe this "lull" further? Can you reproduce this "lull" by
running jobs that any Cygwin user can reproduce?
/Ole
- Re: Problem with thousands of small jobs., (continued)
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/06
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/06
- Re: Problem with thousands of small jobs., Rob Sargent, 2015/02/06
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/06
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/06
- Re: Problem with thousands of small jobs., Ole Tange, 2015/02/06
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/07
- Re: Problem with thousands of small jobs., Ole Tange, 2015/02/07
- Re: Problem with thousands of small jobs., xmoon 2000, 2015/02/10
- Re: Problem with thousands of small jobs., Felipe Alvarez, 2015/02/10
Re: Problem with thousands of small jobs.,
Ole Tange <=