[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[rdiff-backup-users] rdiff-backup millions of small files and memory usa

From: Grégory Pakosz
Subject: [rdiff-backup-users] rdiff-backup millions of small files and memory usage
Date: Thu, 26 Feb 2015 17:17:23 +0100


I'm trying using rdiff-backup to backup an XFS partition which contains millions of small 2K-4K files.
Exactly 63 766 856 files.

I'm using rdiff-backup 1.2.8 on Debian Wheezy with the following packages:
ii  librsync1:amd64                    0.9.7-9
ii  rdiff-backup                       1.2.8-7

I'm using the following options:
--create-full-path --no-hard-links --exclude-other-filesystems --no-file-statistics --verbosity 4

rdiff-backup starts backing up the partition and I can see the memory usage slowly growing. E.g. after running for 1h30 it takes 4GB of RAM according to htop. Ultimately it gets killed after having exhausted the 16GB of available RAM.

Can someone please explain me what data rdiff-backup keeps in memory while backing up?
Any idea on how I can prevent the memory usage to balloon?


reply via email to

[Prev in Thread] Current Thread [Next in Thread]