[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT
From: |
Lasse Collin |
Subject: |
Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT |
Date: |
Mon, 4 Oct 2010 20:58:36 +0300 |
User-agent: |
KMail/1.13.5 (Linux/2.6.35-ARCH; KDE/4.5.1; x86_64; ; ) |
On 2010-10-03 Ralf Wildenhues wrote:
> * Jim Meyering wrote on Sun, Oct 03, 2010 at 02:56:21PM CEST:
> > The name I used there, XZ_OPT, is dictated by the tool. An
> > alternative is to invoke xz with no explicit XZ_OPT setting, but
> > that would result in a semantic change: using the default
> > compression level of -7 rather than what was hard-coded as -9.
The default is -6. In LZMA Utils it is -7, but due to differences in
presets, -6 in XZ Utils is quite similar to -7 in LZMA Utils.
> > Perhaps that would be better, because some people have complained
> > that using -9 makes it harder to decompress (requires more memory)
> > on extremely low-memory (embedded) systems.
Low memory doesn't necessarily imply an embedded device. When
decompressing a source package, it's more likely a desktop computer from
1990s, I think. Yes, some people do use them in 2010.
It's a different question if it makes sense to take these systems into
account when creating source packages. While .xz files can be
decompressed fine on such systems as long as the compression settings
haven't been too high, it may be better to just ask those people to use
.bz2 or .gz version of the source package. The problem is that some
people use a package manager that takes care of downloading the source
files, and it doesn't necessarily support changing which file formats
are preferred.
> > There is no point in changing lzma, since it's deprecated.
>
> Except that the legacy lzma Utils that I tested on one MSYS install
> failed with OOM due to the -9, causing a spurious testsuite failure.
With a recent xz snapshot, it is true with xz too.
> Wrt. xz: I think the general approach for 'dist-*' should be: by
> default use highest compression that is portable. Rationale: the
> default behavior is one compression plus upload, and lots of
> downloads and decompressions. That it makes testing the package
> take longer is a QoI issue which the developer can tune with the
> XZ_OPT knob then. OTOH, when we find out that -9 fails due to OOM
> on some systems (and I have no idea whether the lzma utils failure
> corresponds to a similar xz failure) then we should consider backing
> down to something lower. I really prefer the defaults to work OOTB.
xz used to adjust the compression settings automatically down when the
system had somewhat low amount of RAM, but it has now been disabled as
part of a change affecting decompressor memory usage limiting. I still
don't know if I should restore the limiting & adjusting by default for
compression (and only for that). It currently can be enabled e.g. via
the XZ_DEFAULTS environment variable, but I guess most users won't do
that before something has already gone wrong.
The default level -6 needs 94 MiB of memory. That probably will work
fine for almost everyone. -9 needs 674 MiB, and there will be many users
for which it's too high. Also, -7, -8, and -9 are useful only when the
uncompressed file is bigger than 8 MiB, 16 MiB, and 32 MiB,
respectively. Using a high setting for tiny files is waste of memory. On
the other hand, -e (--extreme) might help with small files too.
--
Lasse Collin | IRC: Larhzu @ IRCnet & Freenode
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, (continued)
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Jim Meyering, 2010/10/03
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Ralf Wildenhues, 2010/10/03
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Jim Meyering, 2010/10/04
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Stefano Lattarini, 2010/10/04
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Jim Meyering, 2010/10/04
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Stefano Lattarini, 2010/10/04
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Jim Meyering, 2010/10/04
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Stefano Lattarini, 2010/10/05
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Ralf Wildenhues, 2010/10/05
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Jim Meyering, 2010/10/05
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT,
Lasse Collin <=
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Ralf Wildenhues, 2010/10/11
- Re: [PATCH] dist-xz: don't hard-code -9: honor setting of XZ_OPT, Lasse Collin, 2010/10/11