Re: [Bug-tar] wishlist: transparent compression on file- and not archive level

[Doug McLaren]

On Wed, Jul 12, 2006 at 10:57:58AM +0200, Christian Hammers wrote:

| For backups, many people decide against the usual compressing method
| via "-j"/"-c" as one single tape/disk medium error makes the complete
| tar archive unusable.

Actually, there's a trivial way around this, and then one that's a bit
more work.

Trivial: simply use bzip2 rather than gzip.  bzip2 compresses data in
block (usually 900 KB in size), and each block is handled
independantly.  If one block is corrupted, the next block will be
unaffected, and decompression can restart there.

Harder: you could make a wrapper around gzip that compresses a few
megabytes of data, then closes the stream, and then re-opens the
stream for another few MB of data, etc.  Then just tell tar to use
this wrapper rather than gzip directly.

For decompression, you'd make your wrapper able to scan through it's
input file so that if a block is corrupted, it can jump to the next
one.

I don't know if anybody has ever written this wrapper.  Personally, I
just use bzip2 for my backups when I want them compessed.  The only
real downside is that bzip2 is a lot slower than gzip.
 
| To circumvent this drawback tar could use libz to zip the individual
| file chunks inside the tar archive so that in the case of a bit error
| on the medium only one file gets corrupted - which would be the case
| in the present implementation anyway.

I suggested something similar about 12 years ago, except that in my
case I was suggesting that if something was to be compressed, it would
just be renamed in the archive -- instead tarring up foo, it would tar
up foo.gz, with the contents of foo.gz being created `on the fly'.  So
this archive _would_ be comptabile with a stock tar, it's just that
you might have to gunzip some files when you're done restoring your
files.  (Of course, it might also be difficult to determine if a file
was compressed on the fly, or was compressed before you started.
Perhaps you coudl use another extension, but then there's still the
same problems ...)

And of course there's the problems that others have brought up ...

-- 
Doug McLaren, address@hidden
If it ain't broke, then fix it 'till it is.