Re: [Duplicity-talk] apparent memory leak on OS X

[Johann Heller]

Hi all,

I'm seeing the same large memory leak on Mac OS X. Details below:

While trying to backup about 1 million files with duplicity I noticed a 
large memory leak. The leak seems to grow depending on the number of 
files processed.

The leak seems to affect versions 0.6.16 and 0.6.17.

Version 0.6.15 works without a leak.

Test numbers below are duplicity doing a full backup of about 10,500 
files using about 350MB of disk space.

The tests are running on a Mac Book Pro with Mac OS X 10.6.8.

I installed duplicity inside a virtualenv with pip using: pip install -v 
http://code.launchpad.net/duplicity/0.6-series/0.6.XX/+download/duplicity-0.6.XX.tar.gz 
(change XX to version number)

The command used is:
ulimit -n 2048;
duplicity \
    --full-if-older-than=12M \
    --encrypt-key=${GPG_KEY} \
    --sign-key=${GPG_KEY} \
    --verbosity=info \
    /Users/johann/Documents/Projects/ file:///Volumes/JHBackup/Documents/

Python 2.6.7 Duplicity 0.6.15 works - mem usage 13.3MB
Python 2.7.2 Duplicity 0.6.15 works - mem usage 12.9MB

Python 2.6.7 Duplicity 0.6.16 leaks - mem usage 35MB
Python 2.7.2 Duplicity 0.6.16 leaks - mem usage 33MB
Python 2.7.2 Duplicity 0.6.17 leaks - mem usage 34MB

Duplicity output for version 0.6.15:
================================================================================
duplicity 0.6.15 (August 19, 2011)
Args: /Users/johann/Documents/Projects/duplicity/bin/duplicity 
--full-if-older-than=12M --encrypt-key=******** --sign-key=******** 
--verbosity=info /Users/johann/Documents/ 
file:///Volumes/JHBackup/Documents/
Darwin ert.local 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 
16:33:36 PDT 2011; root:xnu-1504.15.3~1/RELEASE_I386 i386 i386
/Users/johann/Documents/Projects/duplicity/bin/python2.7 2.7.2 (default, 
Nov 13 2011, 18:43:33)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
================================================================================

Let me know if I can provide any further information or testing.

Thanks,
Johann