Re: Parallel Multicore Usage

[Rob Sargent]

If you want one core per job, you could do 240 jobs simultaneously.  Whether the machines can actually handle the jobs is another question - you say nothing of what is in you’re pipeline and how much multithreading is done within any of the stages.

you need to use the --sshloginfile and associated args

On May 2, 2020, at 9:03 AM, Nyasimi Festus <nfestus@yahoo.com> wrote:

Hi,

I have a few issues in using GNU Parallel.

Problem

I have 150 jobs to run through the same pipeline and each job has different core requirements but utilizes the same pipeline.

I am interested in using GNU parallel to run the jobs in parallel at least 10 at each single time ensuring I utilize all the cores available at my disposal.

Is there a way I have to write my parallel command to ensure I utilize all the cores available to me?

I have 10 nodes each with 24 cores available for this work.

I will really appreciate assistance on this.