Re: [Bug-tar] Solution to updating compressed archives: Pre-compress files

[Tim Kientzle]

On Oct 30, 2011, at 4:45 PM, Carl Eklof wrote:

> Good question, and thanks for getting back to me.
> 
> The nice thing about tar is that it is very serial-device optimized. Though 
> that is because of it's Tape ARchive history, it also makes for every 
> efficient operation for the type of work that we do.
> 
> Are you sure that zip operates as I described? We've experimented with zip, 
> and doing operations such as appending a file resulted in much more disk 
> utilization than our perl-hack, which simply appends onto the end of the 
> file. I guessed that zip must be doing more than just appending onto the end 
> of the file. 

Actually, if all you really want is the ability to append to an existing 
compressed archive, just do it.
  1) gunzip already handles concatenated compressed files.  If you append two 
gzipped files, the output from gunzip will be the concatenation of the 
uncompressed files.
   2) GNU tar already has the hack you need to read such files:  --ignore-zeros

In response to your other questions:
   * Zip updates a directory that tracks the location of each entry.  Part of 
the reason for compressing each entry individually, after all, is to be able to 
extract single items efficiently.
   * Tar does store the length before the contents.
   * There is an official standard for the tar file format:  
http://pubs.opengroup.org/onlinepubs/009695399/utilities/pax.html