[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[rdiff-backup-users] excessive fsync(2)

From: Greg Troxel
Subject: [rdiff-backup-users] excessive fsync(2)
Date: Mon, 31 Oct 2011 13:58:38 -0400
User-agent: Gnus/5.110018 (No Gnus v0.18) Emacs/23.3 (berkeley-unix)

With 1.2.8 on NetBSD, I was seeing insanely long backup times, as in
over a week for a 300G filesystem on which little had changed.
With ktrace, I found the problem was that fsync() was called incredibly
often (per file checked?) and my backup drive was mounted with WAPBL
(metadata journaling), and fsync causes the log to be written, which
causes a cache flush on the drive.  fsync was taking 3/4s most of the

I rebuilt rdiff-backup with the fsync call commented out and then it ran
reasonably quickly).  On another machine with two differences: disk not
mounted with wapbl and rdiff-backup client remote and accessing it over
ssh, either of which might matter, backups go reasonably quickly as

One can argue that this implementation of fsync (cache flush, rather
than some sort of barrier) is inefficient, but fsync cannot properly
return until all the file's data and metadata is safely on disk, so it's
intrinscially a non-streaming expensive operation.

Can anyone explain why rdiff-backup is doing fsyncs so often?

I would expect the plan to be to write all the files and record the
files-written log, and then at the end sync(2) it all (and then wait,
since sync doesn't wait to completion), and then to remove the
backup-in-progress flag.  with the regress-dirty strategy, that kind of
makes the whole thing a transaction.


Attachment: pgpFwUqasrthJ.pgp
Description: PGP signature

reply via email to

[Prev in Thread] Current Thread [Next in Thread]