parallel
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

parallel refusing to take my request


From: Ecks Hecker
Subject: parallel refusing to take my request
Date: Tue, 8 Dec 2020 12:59:05 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1

Hi,

after years of using parallel happily, silently on my private pc, i am
presumeably facing a PEBKAC, and could not resolve it in several days.

Basically, i am building a tree of sparse files resembling the content
of an external device, intending it to be searchable easily with find,
even if is disconnected.

But since there are almost 10000 files, just using find on all of them
gives around 1M of filenames.

To execute a combination of truncate and touch on them on a single core
takes about a minute, which is calling for using parallel (on my 32 cpu
server). But ...

I am running into this (on debian stable):

> parallel: Warning: Only enough file handles to run 252 jobs in parallel.
> parallel: Warning: Running 'parallel -j0 -N 252 --pipe parallel -j0' or
> parallel: Warning: raising 'ulimit -n' or 'nofile' in
> /etc/security/limits.conf
> parallel: Warning: or /proc/sys/fs/file-max may help.
Well, i did raise the 'nofile' limit as indicated, even tried the --pipe
construct, to no avail.

ulimit -n gives 1024

Apart from that, there seems to be some misunderstanding from my side,
as i am going round in circles without making progress. Any hints would
be greatly appreciated. Apparently, this one is above my paygrade and
skill level. :-(

regards, Ecks




reply via email to

[Prev in Thread] Current Thread [Next in Thread]