Re: [rdiff-backup-users] --check-destination-dir taking a very long time

[Patrik Dufresne]

For a server, I would recommend ZFS without hesitation. Data integrity is
at the heart of ZFS which makes it a very good choice for backup storage.
It provides many features that ease the storage management. Quota,
compression, configurable record size, configurable sync writes, raidz,
snapshots, etc.


For an external storage, like a USB drive, I would probably pick BTRFS for
interchangeability reason. ZFS might still be used for an external drive,
but required the installation of ZFS on Linux to be working which is not
installed by default. While BTRFS is shipped as part of Linux Kernel. So in
case of emergency, you might plug the disk in almost any Linux computer and
get access to your files. BTRFS provide checksum and copy on write feature
that you want to ensure data integrity. But I would not recommend BTRFS for
any RAID setup. RAID5/6 is known to be buggy for a couple of years. I would
not trust my data on a BTRFS RAID. But for a single drive it's a very good
alternative to ext4

--
Patrik Dufresne Service Logiciel inc.
http://www.patrikdufresne.com <http://patrikdufresne.com/>/
514-971-6442
130 rue Doris
St-Colomban, QC J5K 1T9


On Thu, Sep 12, 2019 at 8:38 AM Harald Hannelius <address@hidden>
wrote:

>
>
> What would You recommend instead?
>
> On Thu, 12 Sep 2019, Patrik Dufresne wrote:
>
> > Hello Walt,
> >
> > That's a good news you manage to recover your backup.
> >
> > ext4 has limitation that might be reached for a backup storage.
> Personally,
> > I would way ext4 file system is not a good choice for data storage. As
> you
> > found, it has limitation on number of subfolder, it's a journaling
> > filesystem, doesn't have copy on write, not snapshot, it doesn't have any
> > checksum either.
> >
> > --
> > Patrik Dufresne Service Logiciel inc.
> > http://www.patrikdufresne.com <http://patrikdufresne.com/>/
> > 514-971-6442
> > 130 rue Doris
> > St-Colomban, QC J5K 1T9
> >
> >
> > On Thu, Sep 12, 2019 at 7:53 AM Walt Mankowski <address@hidden>
> wrote:
> >
> >> Update -- it took about 30 hours to do the regression, started doing
> >> the backup, and then quickly crashed with another "No space left on
> >> device" exception. In kern.log I found these messages:
> >>
> >> Sep 11 21:53:47 scruffy kernel: [294433.260536] EXT4-fs warning (device
> >> sde1): ext4_dx_add_entry:2190: Directory (ino: 30017692) index full,
> reach
> >> max htree level :2
> >> Sep 11 21:53:47 scruffy kernel: [294433.260540] EXT4-fs warning (device
> >> sde1): ext4_dx_add_entry:2194: Large directory feature is not enabled on
> >> this filesystem
> >>
> >> It got the error while trying to write a file in
> >> ~/.cache/chromium/Default/Cache. Turns out that after running
> >> rdiff-backup for years I'd accumulated over 6 million files in
> >>
> >>
> /backup/scruffy/rdiff-backup-data/increments/home/waltman/.cache/chromium/Default/Cache
> >> It's a bit fuzzy how many files you can add to an ext4 directory, but
> >> it looks like I hit the limit.
> >>
> >> I added ~/.cache/chromium to my exclude list and restarted the
> >> backup. It looks a lot happier now.
> >>
> >> Walt
> >>
> >> _______________________________________________
> >> rdiff-backup-users mailing list at address@hidden
> >> https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> >> Wiki URL:
> >> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
> > _______________________________________________
> > rdiff-backup-users mailing list at address@hidden
> > https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
> > Wiki URL:
> http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
> >
> >
>
> --
>
> Harald Hannelius | harald.hannelius/a\arcada.fi | +358 50 594 1020
>