[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [rdiff-backup-users] Incredibly slow i/o to NAS server

From: Robert Nichols
Subject: Re: [rdiff-backup-users] Incredibly slow i/o to NAS server
Date: Tue, 29 Nov 2016 11:16:12 -0600
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0

On 11/28/2016 07:31 PM, Andrea Bolandrina wrote:

I'm running an rdiff-backup script, to backup my laptop to my local NAS.

I love rdiff-backup and it normally works great, but at the moment I'm having 
problem with one specific directory.
Such directory is where is store my docker images, therefore it's a lot of 
To be precise:
sudo find /mnt/vms/docker/ -type f|wc -l
sudo du -hs /mnt/vms/docker
4.9G    /mnt/vms/docker
So, it's nearly a million files, but less than 5GB.

When I run the backup the first time, it didn't take too long (can't remember 
how long exactly, but less than a couple of hours).
Then I've updated some docker images, and removed the old ones (clearly lots of 
Now I re-run the rdiff-backup and it has been running for nearly 24 hours on 
that specific folder, no problem with all the other folders.

First, note that the files (except for those with a suffix known to be 
uncompressible) in those removed images is going to be compressed and stored as 
a .snapshot.gz file. That can take quite a while. Even something as simple as 
deleting an old kernel adds 15 minutes or so to my next backup as all the 
deleted files under /usr/src/kernels and /lib/modules get compressed and saved 

Second, are there a large number of files with more than one hard link ("find /mnt/vms/docker/ -type f -links +1 | 
wc -l")? Those can be an issue if the device number (st_dev) changes ("Device:" in the output from the 
"stat" command). When that happens, every one of those multi-link files will be seen as changed and have a 
zero-diff file created in the increments directory. You can avoid that by using the "--no-compare-inode" 
option, but doing that aggravates the problems that rdiff-backup already has with hard-linked files.

Bob Nichols     "NOSPAM" is really part of my email address.
                Do NOT delete it.

reply via email to

[Prev in Thread] Current Thread [Next in Thread]