Re: [Duplicity-talk] OSError: [Errno 24] Too many open files

[Diagon]

Date: Mon, 20 Jul 2020 11:48:04 +0200

> On 20.07.2020 07:16, Diagon via Duplicity-talk wrote:
>> Ubuntu 16.04, and yes I'm still running 0.7.19. Everything was fine until my 
>> cron scheduled backups stopped without warning on June 6th, as I just 
>> discovered.
>>
>> Command line that looks something like:
>>
>> PASSPHRASE="xxx" duplicity --log-file /home/me/duplicity.log 
>> --backend-retry-delay 60 --asynchronous-upload --name Remote --volsize 50 
>> --full-if-older-than 6M --exclude '**.lock' /home/dev/mydir 
>> sftp://me@server.com/Backup
>>
>> Error below. Any way around this until I get a new OS installed?
>>
>> Thanks -
>>
>> /usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py:141: 
>> FutureWarning: CTR mode needs counter parameter, not IV
SNIP
>> . File "/usr/lib/python2.7/dist-packages/duplicity/gpginterface.py", line 
>> 374, in run
> >. create_fhs, attach_fhs)
>> . File "/usr/lib/python2.7/dist-packages/duplicity/gpginterface.py", line 
>> 402, in _attach_fork_exec
>> . pipe = os.pipe()
>> . OSError: [Errno 24] Too many open files

> for now you can work around the issue by raising the open files limit via 
> ulimit
> https://linuxhandbook.com/ulimit-command/

Is that on the server or on the client that I have to raise that limit?

> would you mind posting a 'duplicity collection-status ...' of your backup. i 
> would guess you some really very very long chain in there. maybe it is time 
> to do a new full?

I'm backing up this directory every 20 minutes, but almost all of the time it's 
only one file changing.  So it looks like the following:

$ duplicity collection-status sftp://me@server.com/Backup
/usr/lib/python2.7/dist-packages/Crypto/Cipher/blockalgo.py:141: FutureWarning: 
CTR mode needs counter parameter, not IV
  self._cipher = factory.new(key, *args, **kwargs)
  Last full backup date: Wed May 13 22:20:03 2020
  Collection Status
  -----------------
  Connecting with backend: BackendWrapper
  Archive dir: /home/me/.cache/duplicity/xxxxxxxxxxxxxxxxxxxxxx

Found 0 secondary backup chains.

Found primary backup chain with matching signature chain:
-------------------------
Chain start time: Wed May 13 22:20:03 2020
Chain end time: Mon Jun 15 17:40:06 2020
Number of contained backup sets: 1007
Total number of contained volumes: 1052
Type of backup set:                            Time:      Num volumes:
Full                Wed May 13 22:20:03 2020                46
Incremental         Thu May 14 03:40:04 2020                 1

<That last line, with different date/times and Num volumes = 1, repeats 1006 
times>

> ..ede/duply.net

/D