Re: [Duplicity-talk] apparent memory leak on OS X

[Ken Bass]

On 1/9/2012 10:16 AM, Kenneth Loafman wrote:

On Mon, Jan 9, 2012 at 2:38 AM, Michael Terry <address@hidden> wrote:

On 6 January 2012 19:45, Nate Eldredge <address@hidden> wrote:
> To further narrow this down, I tried some different revisions in the
> lp:duplicity branch.  It appears that revision 782 is good but 783 has the
> bug.  The relevant change is labeled
>
> "Merged in lp:~mterry/duplicity/tarfile"

Ah, that's me!  :)  Thanks for tracking it down that far.
Unfortunately, that was a rather big merge.

I'm happy to look into this, but I don't have much experience with
debugging leaks in Python.  Are there good tools or techniques for
that?

Try this.  A friend of mine recommended it.

http://www.lshift.net/blog/2008/11/14/tracing-python-memory-leaks

One take away from that blog article is that:

"The garbage collector will collect objects in reference cycles unless they have del method."

A quick grep through the code shows that tempfile, tarfile, and pexpect each have __del__ method.
Since you said it happens with large datasets, perhaps it has to do with either a large number of tar or tempfiles.

And it just so happens that mterry added a __del__ method to tarfile.py in that patch.

Related?