Re: [Bug-tar] Cannot restore incremental backup after directories deleted and renamed

[Joerg Schilling]

"Dieterly, Deklan" <address@hidden> wrote:

> This script produces an error. "tar: Cannot rename 'backup/dir1' to 
> 'backup/dir2': Directory not empty"
>
> This is a use case that we would like to be able to handle. I've seen other 
> threads describe this problem too.
>
> Here are the links to the other threads.
>
> https://lists.gnu.org/archive/html/bug-tar/2008-07/msg00005.html
> https://bugs.launchpad.net/freezer/+bug/1570304
> http://osdir.com/ml/bug-tar-gnu/2011-11/msg00016.html
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=648048

As you may see from the last entry, I reported this bug in September 2004 
already.

> Here is the script that produces the error.
...
> tar: Cannot rename 'backup/dir1' to 'backup/dir2': Directory not empty
> tar: Exiting with failure status due to previous errors
>
>
> Will this problem be addressed and fixed, or is there some fundamental 
> underlying reason why the problem will not be fixed? Thanks.

Given that the problem is known since nearly 12 years and there is no fix, I 
guess that the gtar maintainers also believe that there is no way to fix the 
problem without introducing a new incompatible archive format.


Let me start with a modified version of what I wrote in 2011:

This is a problem that I reported in 2004 already. The problem was discovered 
in September 2004 while running the first test for the new incremental 
dump/restore format in star. The tests had been made with ufsdump/ufsrestore, 
gtar and star. Gtar was the only program that completely failed in this test, 
so I am wondering why there are still backup management programs that use gtar 
as their backend.

I am not sure whether it is possible to solve the problem without introducing a 
new incompatible dump format in GNU tar. AFAIK Gnu tar tries to detect and 
understand all changes while creating the archive by using a partial database 
during create. Star detects and understands changes at extract time by using a 
complete database which coveres all files.

Star with it's incremental dump format that was inspired by ufsdump is able to 
handle all known deltas on a filesystem. Star (as "stable version") is 
available 
at:

        https://sourceforge.net/projects/s-tar/files/

Frequent development snapshots are inside the "schilytools" tarball at:

        https://sourceforge.net/projects/schilytools/files/


Star was running an incremental dump + ** incremental restore ** once a day on 
berlios.de with not a single problem since March 2005 (for aprox. 10 years 
until 
a new ZFS based fileserver with a snapshot based backup was introduced). There 
have been typically of 2-10 GB of changes per day on thousands of files and more
than 3500 successful dump/restore operations have been made in incremental mode.

Between September 2004 (the initial release of the star incremental support) 
and Mach 2005, there was one single minor problem that had to be fixed.


The difference between star and gtar is that star uses a known to work 
algorithm and archives all needed meta data inside the archive. When you do an 
incremental restore, star creates a database of filenames together with old and 
new inode numbers that permit to track any rename operation.

Gtar has no real support for tracking renames. Gtar just fully archives any 
renamed object. So if you have a 1TB disk with 900GB of data in a single top 
level directory and rename that directory, the next incremental holds 900GB of
gtar archive data.

If you try to do an incremental restore on that data, you would need 2x 900GB 
of space on the target filesystem.

If you do the same using star, the incremental archive holds 10kB and the 
incremental restore does not need more free space on the disk than you had in 
the original. If files are removed, this happens at the end of an incremental
restore operation and you need sufficient space to hold the removed files and 
the new files, but this is what you need with gtar as well.

Jörg

-- 
 EMail:address@hidden                    (home) Jörg Schilling D-13353 Berlin
       address@hidden (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.org/private/ 
http://sourceforge.net/projects/schilytools/files/'