[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Large files truncated using xorriso 1.5.4 using zisofs v2

From: Thomas Schmitt
Subject: Re: Large files truncated using xorriso 1.5.4 using zisofs v2
Date: Thu, 10 Nov 2022 15:56:35 +0100


Dag Wieers wrote:
> The resulting image was 300MB smaller (about 1.5%)

That's understandable because zisofs individually compresses chunks of
32, 64, or 128 KiB in order to achieve coarse random access.
The --gzip filter compresses the whole file in one sweep.

> but, more importantly,
> generating the ISO image was almost twice as fast (30 mins vs 50 mins) !

That's strange.

I lack of realisticly compressible large tarballs. A test with a 5.4 GB
ISO image as payload file and /dev/null as output drive on 3.5 GHz Xeon
with enough RAM to hold the whole payload yields the contrary result:
  --gzip   : 258 s
  --zisofs : 235 s

The compression by xorriso on-the-fly needs two compression passes.
One at image planning time, to determine the compressed size, and one at
image write time, to produce the compressed stream.
If compression time is a problem, then consider to compress the tarballs
already on hard disk.
gzipping above payload file into /dev/null needs 132 s.

Differences between --zisofs and --gzip in regard to usage of zlib:

zisofs uses zlib function compress2() whereas the gzip filter of
libisofs uses zlib function deflate().

According to /usr/include/zlib.h the function compress2() is a variation
of compress(), where the compression level is an additional parameter.
About both the include file says:
  "The following utility functions are implemented on top of the basic
   stream-oriented functions."

Mark Adler (probably one of the copyright holders) states
  "You would use compress() if you have all the data available at once and
   enough memory to hold the result, and you want the default compression
   format, memory usage, and strategy. Otherwise, you would use deflate()."

So the only reason i can imagine for zisofs compression being so much
slower is the initialization of the compression context for each chunk of
data. But that would be quite independent of the payload's compressibility
and should be to see in my own measurements.

Have a nice day :)


reply via email to

[Prev in Thread] Current Thread [Next in Thread]