Re: [Duplicity-talk] OSError: [Errno 24] Too many open files

[Kenneth Loafman]

Yang Zhang wrote:
> Kenneth Loafman wrote:
>> Yang Zhang wrote:
>>> I'm running into this exception (running on Ubuntu 9.04). Anybody know
>>> what's up with this?
>>>
>>> $ uname -a
>>> Linux yang-xps410 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:58:03
>>> UTC 2009 x86_64 GNU/Linux
>>>
>>> $ duplicity --version
>>> duplicity 0.5.17
>>>
>>> $ duplicity /home/yang/personal/
>>> OSError: [Errno 24] Too many open files
>>
>> This line says it all.  I just looked and Jaunty has the default of 1024
>> Max Open Files.  duplicity will have open up about 150 of those at any
>> one time and each task running will have their own files open, often 50
>> or more, so its easy to deplete the resource.
>>
>> The fix is easy: "sudo ulimit -n 4096" will up the count to 4096 Max
>> Open Files.  This should solve the problem.
>>
>> ...Ken
> 
> But this limit is per-process, not a shared pool across all processes.
> It means *duplicity* (via GPG, apparently) is opening more than 1024
> files at a time.
> 
> See also this closed bug:
> 
> http://savannah.nongnu.org/bugs/?21792

It's possible there's a bug in gpg that's keeping files open on Jaunty.
 I don't have that system to check against, so if you would, please do:

1) run duplicity in a fairly long job, or with a small --volsize
2) while its running, do:
     # lsof -n | grep ^duplicity
and save the last output you get.  I think it will show the same problem
as the other bugs have, lots of gpg processes not closed.  I thought I
had this problem fixed.   If that's not the case, then zip the listing
and post it to the bug report above.  I'll reopen it.

...Ken

signature.asc
Description: OpenPGP digital signature