Re: Large files truncated using xorriso 1.5.4 using zisofs v2

[Thomas Schmitt]

Hi,

Dag Wieers wrote:
> The resulting image was 300MB smaller (about 1.5%)

That's understandable because zisofs individually compresses chunks of
32, 64, or 128 KiB in order to achieve coarse random access.
The --gzip filter compresses the whole file in one sweep.


> but, more importantly,
> generating the ISO image was almost twice as fast (30 mins vs 50 mins) !

That's strange.

I lack of realisticly compressible large tarballs. A test with a 5.4 GB
ISO image as payload file and /dev/null as output drive on 3.5 GHz Xeon
with enough RAM to hold the whole payload yields the contrary result:
  --gzip   : 258 s
  --zisofs : 235 s

The compression by xorriso on-the-fly needs two compression passes.
One at image planning time, to determine the compressed size, and one at
image write time, to produce the compressed stream.
If compression time is a problem, then consider to compress the tarballs
already on hard disk.
gzipping above payload file into /dev/null needs 132 s.

--------------------------------------------------------------------
Differences between --zisofs and --gzip in regard to usage of zlib:

zisofs uses zlib function compress2() whereas the gzip filter of
libisofs uses zlib function deflate().

According to /usr/include/zlib.h the function compress2() is a variation
of compress(), where the compression level is an additional parameter.
About both the include file says:
  "The following utility functions are implemented on top of the basic
   stream-oriented functions."

In
  
https://stackoverflow.com/questions/10166122/zlib-differences-between-the-deflate-and-compress-functions
Mark Adler (probably one of the copyright holders) states
  "You would use compress() if you have all the data available at once and
   enough memory to hold the result, and you want the default compression
   format, memory usage, and strategy. Otherwise, you would use deflate()."

So the only reason i can imagine for zisofs compression being so much
slower is the initialization of the compression context for each chunk of
data. But that would be quite independent of the payload's compressibility
and should be to see in my own measurements.


Have a nice day :)

Thomas