Re: [Duplicity-talk] apparent memory leak on OS X

[Nate Eldredge]

On Wed, 21 Dec 2011, Johann Heller wrote:

> Hi all,
>
> I'm seeing the same large memory leak on Mac OS X. Details below:
>
> While trying to backup about 1 million files with duplicity I noticed a large 
> memory leak. The leak seems to grow depending on the number of files 
> processed.
>
> The leak seems to affect versions 0.6.16 and 0.6.17.
>
> Version 0.6.15 works without a leak.

I'm seeing this as well, on Ubuntu Natty i386.  Duplicity 0.6.16 and 
0.6.17 are affected while 0.6.15 is fine.

Note that the effect is much more dramatic with a large number of files, 
100,000 or more.  It also seems to be even worse for incremental backups.

My regular backup, which has about 450,000 files, was not able to 
complete.  The machine ran out of memory (it has 2G physical and about 3G 
swap) and the oom-killer killed duplicity.  When I added an extra 10G of 
swap, duplicity was eventually able to finish, but it took about 10 times 
longer than usual and made the machine unusable in the meantime (due to 
excessive swapping).  So this is pretty serious as far as I am concerned.

Bug #908228 seems to describe the same issue.

--

Nate Eldredge
address@hidden