[Duplicity-talk] A couple of checkpoint/restore related bugs

[Liraz Siri]

I've been testing Duplicity's resume functionality and I may have
discovered a couple of bugs. FWIW, I'm using duplicity 0.6.09 installed
on a 32-bit Ubuntu system, backing up to the local filesystem.

What I'm doing is invoking duplicity like this:

duplicity --archive-dir /var/cache/duplicity --volsize 50
--include-filelist /tmp/filelist --exclude '**' /
file:///var/tmp/duplicity

And then CTRL-C after some time has passed, and try to resume by
invoking the same command.

bug #1: CTRL-C after last volume is created but before signature
================================================================

If you hit CTRL-C after the last volume is created but before the
signature file, trying to invoke duplicity again will fail with two
kinds of errors.

type 1 error
------------

# exception raised after CTRL-C
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1222, in main
    full_backup(col_stats)
  File "/usr/bin/duplicity", line 421, in full_backup
    sig_outfp.to_final()
  File "/usr/lib/python2.5/site-packages/duplicity/dup_temp.py", line
195, in to_final
    gpg.GzipWriteFile(src_iter, tgt.name, size = sys.maxint)
  File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 335, in
GzipWriteFile
    gzip_file.write(new_block.data)
  File "/usr/lib/python2.5/gzip.py", line 205, in write
    self.fileobj.write( self.compress.compress(data) )
KeyboardInterrupt

# exception raised after attempted resume
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1149, in main
    globals.archive_dir).set_values()
  File "/usr/lib/python2.5/site-packages/duplicity/collections.py", line
687, in set_values
    backup_chains)
  File "/usr/lib/python2.5/site-packages/duplicity/collections.py", line
700, in set_matched_chain_pair
    sig_chains = sig_chains and self.get_sorted_chains(sig_chains)
  File "/usr/lib/python2.5/site-packages/duplicity/collections.py", line
913, in get_sorted_chains
    assert len(chain_list) == 2
AssertionError

type 2 error
------------

# CTRL-C raises a GPGError. No traceback
GPGError: GPG Failed, see log below:
===== Begin GnuPG log =====
===== End GnuPG log =====

# attempted resume

Deleting local
/var/cache/duplicity/6ba373041a8ebe6cdcdbe79e5d16ad38/duplicity-full-signatures.20100729T034328Z.sigtar.gz
(not authoritative at backend).
Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1145, in main
    sync_archive()
  File "/usr/bin/duplicity", line 957, in sync_archive
    remove_local(fn)
  File "/usr/bin/duplicity", line 872, in remove_local
    os.unlink(del_name)
OSError: [Errno 2] No such file or directory:
'/var/cache/duplicity/6ba373041a8ebe6cdcdbe79e5d16ad38/duplicity-full-signatures.20100729T034328Z.sigtar.gz'

bug #2: CTRL-C before first volume - temporary files not cleaned up
===================================================================

If I CTRL-C before the first volume is completed, temporary files
created in the cache are not cleaned up. Clues may be found in the
exception raised on CTRL-C:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1251, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1244, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1226, in main
    incremental_backup(sig_chain)
  File "/usr/bin/duplicity", line 487, in incremental_backup
    globals.backend)
  File "/usr/bin/duplicity", line 294, in write_multivol
    globals.gpg_profile, globals.volsize)
  File "/usr/lib/python2.5/site-packages/duplicity/gpg.py", line 279, in
GPGWriteFile
    data = block_iter.next(min(block_size, bytes_to_go)).data
  File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line
505, in next
    result = self.process(self.input_iter.next(), size)
  File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line
187, in get_delta_iter
    for new_path, sig_path in collated:
  File "/usr/lib/python2.5/site-packages/duplicity/diffdir.py", line
265, in collate2iters
    relem1 = riter1.next()
  File "/usr/lib/python2.5/site-packages/duplicity/selection.py", line
174, in Iterate
    subpath, val = diryield_stack[-1].next()
  File "/usr/lib/python2.5/site-packages/duplicity/selection.py", line
141, in diryield
    for filename in robust.listpath(path):
  File "/usr/lib/python2.5/site-packages/duplicity/robust.py", line 61,
in listpath
    dir_listing = check_common_error(error_handler, path.listdir)
  File "/usr/lib/python2.5/site-packages/duplicity/robust.py", line 37,
in check_common_error
    return function(*args)
  File "/usr/lib/python2.5/site-packages/duplicity/path.py", line 514,
in listdir
    return os.listdir(self.name)
KeyboardInterrupt
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound method TemporaryDirectory.__del__ of
<duplicity.tempdir.TemporaryDirectory instance at 0x8922dac>> ignored
Exception exceptions.TypeError: "'NoneType' object is not callable" in
<bound method TemporaryDirectory.__del__ of
<duplicity.tempdir.TemporaryDirectory instance at 0x8922eac>> ignored

Cheers,
Liraz