Re: [Duplicity-talk] High memory usage with large number of files

[Chris Adamson]

I investigated and the issue is with os.listdir

see:

http://www.olark.com/spw/2011/08/you-can-list-a-directory-with-8-million-files-but-not-with-ls/ 

I think I need to not have that many files in one directory.

On Thu, Jan 19, 2012 at 8:08 PM, <address@hidden> wrote:

On 19.01.2012 06:04, Chris Adamson wrote:
> Hi List,
>
> I wish to backup a ~4.5TB file server and duplicity used > 25GB of RAM on a
> directory with 477259 files. What debugging steps can I take to diagnose
> the problem?
>

sounds like this issue.
http://lists.gnu.org/archive/html/duplicity-talk/2012-01/msg00005.html

..ede/duply.net

_______________________________________________
Duplicity-talk mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/duplicity-talk