|
From: | FUNG Yik Him Zachary |
Subject: | Re: Properly scheduling multi-threaded/process jobs |
Date: | Thu, 2 Sep 2021 04:56:41 +0000 |
Hi,
I come with a situation that the inputs of my application vary in a wide range such that their running speed could swing from I/O-bounded to computation-bounded, and therefore there are different efficient numbers of
CPU cores to be used in different jobs. It would be also nice to me if GNU parallel would be able aware of the job size (job slots to be occupied by each job). I try to use GNU parallel instead of queue systems because there are nice data handling features “--basefile/--return” and spamming the cluster queue with thousands of small tasks is not appreciated by the cluster’s admin.
That’s why I am echoing this old idea =] Zachary |
[Prev in Thread] | Current Thread | [Next in Thread] |