[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Lzip-bug] lzip memory & performance issue

From: Antonio Diaz Diaz
Subject: Re: [Lzip-bug] lzip memory & performance issue
Date: Thu, 24 Feb 2011 17:48:52 +0100
User-agent: Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.11) Gecko/20050905

Hello Mark,

address@hidden wrote:
lzip's initial memory check seems to be too conservative. Or does lzip
uses more memory than xz and lzma_alone with the same dictionary size?

IIRC lzip 1.11 does use a few MB more than lzma-utils for the same dictionary size. I don't know the amount used by other lzma_alone implementations.

Lzip 1.11 does not perform any memory check. It tries to assign the needed memory, and if it can't, it prints the "Not enough memory. Try a smaller dictionary size." message. I think xz does perform a memory check and adjusts the dictionary size accordingly.

From http://www.nongnu.org/lzip/manual/lzip_manual.html#Introduction
"The amount of memory required for compression is about 5 MiB plus 1 or 2 times the dictionary size limit (1 if input file size is less than dictionary size limit, else 2) plus 8 times the dictionary size really used. The option `-0' is special and only requires about 1.5 MiB at most."

So for that specific file, lzip took about 2.5 times as long as xz.

I guess your system is swapping slightly because of the large dictionary size. As you can see, for example here[1], lzip and xz are comparable both in compression time and size, with "lzip -9" tending to be slower than "xz -9" but compressing a little more.
[1] http://martin-steigerwald.de/computer/programme/packbench/index.html

Any idea what causes the time difference? Could
different compiler options have that large an effect? I think I compiled
xz with -O3 but lzip with -O2.

I have only noticed small speed changes in lzip between -O2 and -O3. I don't know the effects on xz, but I don't think they can be so large.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]