Re: Option wanted for gzip

[Bob Proulx]

Raoul Behrend wrote:
> For the application I devellope, I have to collect gzipped astronomical 
> images, calibrate (after gunzipping) them, and archive the raw image in a 
> compressed manner. The new modified ones are also archived after a 
> gzipping, while a copy of the unzipped processed images are sent to a 
> second pass of reduction.

I think I followed that.  The came in gzip'd.  Long term archive
gzip'd.  Also keep a plain copy for current work, like a cache.

> So, the flux is as below.
> raw.fit.gz  - archived
>             - gunzipped - modified by first pass
>                   - gzipped and archived
>                   - second pass of processing
> 
> The first and second pass usually are not done at the same moment, as some 
> data have to be collected and analysed before the second pass. Sometimes, 
> two ot three passes of the whole files in necessary.

Okay.

> Due to the huge size of the data on the disks, I 'd like to keep to a 
> minimum hard disk works: this can be done using a kind of keeping of the 
> entry file in its current state at the end of the decompression or 
> compression operations.

Sure.  But then can't you just do this already?

  gzip -dc < raw.fit.gz > raw.fit

That is the same as:

  gunzip < raw.fit.gz > raw.fit

> bzip2 had a -k option which do the work nicely.

Sure a -k option would work.  But please explain how the above
examples shown do not work.  No change to the C source is required.

> Remark: the naming of the file is unpredictable, so a special unportable 
> macro should be developped if one want to use the < and > redirectors to 
> specify the output name, as suggested in your first answer.

I think this is something that you know that we do not know.  Please
educate us!  You know something about your problem that makes this
hard.  But to me using shell for this is easy.  Here is an example

  #!/bin/sh
  filename=raw.fit
  gzip -dc < $filename.gz > $filename

Or

  #!/bin/sh
  filename=raw.fit.gz
  gzip -dc < $filename > $(basename $filename .gz)

With the -k option it would be similar.

  #!/bin/sh
  filename=raw.fit.gz
  gzip -dk $filename

People apparently want a gzip -k option and so eventually will
probably get it.  But meanwhile I maintain that it is not needed.  The
capability exits without needing an option.  Personally I would avoid
it because it will take years after addition before it could be
counted upon as available for use.

Bob