[rdiff-backup-users] Error: Too many open files

[Jérémy JUST]

    Hi,

  I've been using rdiff-backup for a month, but since a few days,
it fails with that error:

<<<<<
Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 23, in ?
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 259, in 
Main
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 229, in 
take_action
  File "/usr/lib/python2.3/site-packages/rdiff_backup/Main.py", line 278, in 
Backup
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 51, in 
Mirror_and_increment
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 226, in 
patch_and_increment
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rorpiter.py", line 279, 
in __call__
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 607, in 
fast_process
  File "/usr/lib/python2.3/site-packages/rdiff_backup/backup.py", line 593, in 
inc_with_checking
  File "/usr/lib/python2.3/site-packages/rdiff_backup/increment.py", line 44, 
in Increment
  File "/usr/lib/python2.3/site-packages/rdiff_backup/increment.py", line 73, 
in makesnapshot
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 213, in 
copy_with_attribs
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 96, in 
copy
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 118, in 
copy_reg_file
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 946, in 
write_from_fileobj
  File "/usr/lib/python2.3/site-packages/rdiff_backup/rpath.py", line 925, in 
open
  File "/usr/lib/python2.3/gzip.py", line 94, in __init__
IOError: [Errno 24] Too many open files: 
'/mnt/usb/rdiff-backup-data/increments/var/spool/news/mess
age.id/065/<address@hidden>.2005-01-13T22:08:59+01:00.snapshot
.gz'
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del
__ of <gzip open file 
'/mnt/usb/rdiff-backup-data/file_statistics.2005-01-24T23:16:32+01:00.data.gz
', mode 'wb' at 0x40491060 0x40841ccc>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del
__ of <gzip open file 
'/mnt/usb/rdiff-backup-data/error_log.2005-01-24T23:16:32+01:00.data.gz', mod
e 'wb' at 0x4048d4a0 0x409a276c>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in <bound 
method GzipFile.__del
__ of <gzip open file 
'/mnt/usb/rdiff-backup-data/mirror_metadata.2005-01-24T23:16:32+01:00.snapsho
t.gz', mode 'wb' at 0x404914e0 0x4084140c>> ignored
>>>>>


  The concerned directory is the news spool, managed by Leafnode.
« /var/spool/news/message.id/065/ » currently contains only about
140 files.

  I've monitored opened files during rdiff-backup run, with lsof.
These are the counts:         (interval = 5 seconds)
 51  52  53  52  92  83  117  186  364  522  717  876  1040  0  0  0  0
                                                           ^
                                                         crash

  Just before the crash, I can see about one thousand files opened
on the backup partition, in various directories:

<<<<<
/mnt/usb/var/spool/news/message.id/051/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/051/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/051/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/051/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/051/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/052/<address@hidden> (deleted)
    [...]
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/056/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/057/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/057/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/057/<address@hidden> (deleted)
/mnt/usb/var/spool/news/message.id/057/<address@hidden> (deleted)
>>>>>


  I'm using rdiff-backup 0.13.4 with Python 2.3.
  My ulimit for opened files is set at 1024.


  Is that a known problem of the development version? Should I downgrade
to 0.12.7? (but it seems that backup formats are incompatible)


Cheers.

-- 
Jérémy JUST  <address@hidden>