[Rdiff-backup-bugs] [bug #38325] --check-destination-dir memory required is not constant

[Niccolo Rigacci]

Follow-up Comment #1, bug #38325 (project rdiff-backup):

Hi,

the command:
rdiff-backup --check-destination-dir /backup/hostname
fails after consuming over 4Gb of real memory (it used zero swap memory).
There are about 13 million files (709 Gb) of data.

Here the traceback (verson 1.2.8 on Debian Squeeze):

Traceback (most recent call last):
  File "/usr/bin/rdiff-backup", line 30, in <module>
    rdiff_backup.Main.error_check_Main(sys.argv[1:])
  File "/usr/lib/pymodules/python2.6/rdiff_backup/Main.py", line 304, in
error_check_Main
    try: Main(arglist)
  File "/usr/lib/pymodules/python2.6/rdiff_backup/Main.py", line 324, in Main
    take_action(rps)
  File "/usr/lib/pymodules/python2.6/rdiff_backup/Main.py", line 282, in
take_action
    elif action == "check-destination-dir": CheckDest(rps[0])
  File "/usr/lib/pymodules/python2.6/rdiff_backup/Main.py", line 872, in
CheckDest
    dest_rp.conn.regress.Regress(dest_rp)
  File "/usr/lib/pymodules/python2.6/rdiff_backup/regress.py", line 71, in
Regress
    for rf in iterate_meta_rfs(mirror_rp, inc_rpath): ITR(rf.index, rf)
  File "/usr/lib/pymodules/python2.6/rdiff_backup/regress.py", line 197, in
iterate_meta_rfs
    for raw_rf, metadata_rorp in collated:
  File "/usr/lib/pymodules/python2.6/rdiff_backup/rorpiter.py", line 100, in
Collate2Iters
    try: relem2 = riter2.next()
  File "/usr/lib/pymodules/python2.6/rdiff_backup/eas_acls.py", line 207, in
join_ea_iter
    for rorp, ea in rorpiter.CollateIterators(rorp_iter, ea_iter):
  File "/usr/lib/pymodules/python2.6/rdiff_backup/rorpiter.py", line 103, in
Collate2Iters
    for relem1 in riter1:
  File "/usr/lib/pymodules/python2.6/rdiff_backup/eas_acls.py", line 566, in
join_acl_iter
    for rorp, acl in rorpiter.CollateIterators(rorp_iter, acl_iter):
  File "/usr/lib/pymodules/python2.6/rdiff_backup/rorpiter.py", line 103, in
Collate2Iters
    for relem1 in riter1:
  File "/usr/lib/pymodules/python2.6/rdiff_backup/metadata.py", line 274, in
iterate
    for record in self.iterate_records():
  File "/usr/lib/pymodules/python2.6/rdiff_backup/metadata.py", line 283, in
iterate_records
    next_pos = self.get_next_pos()
  File "/usr/lib/pymodules/python2.6/rdiff_backup/metadata.py", line 266, in
get_next_pos
    newbuf = self.fileobj.read(self.blocksize)
  File "/usr/lib/python2.6/gzip.py", line 219, in read
    self._read(readsize)
  File "/usr/lib/python2.6/gzip.py", line 272, in _read
    self._add_read_data( uncompress )
  File "/usr/lib/python2.6/gzip.py", line 289, in _add_read_data
    self.extrabuf = self.extrabuf + data
MemoryError

    _______________________________________________________

Reply to this item at:

  <http://savannah.nongnu.org/bugs/?38325>

_______________________________________________
  Messaggio inviato con/da Savannah
  http://savannah.nongnu.org/