Re: Please, use --check=crc32 or switch to a safe format

[Antonio Diaz Diaz]

Michael Stone wrote:
> On Wed, Apr 05, 2017 at 04:16:53PM -0300, Matias Fonzo wrote:
> > There's nothing wrong in want to provide a "second" format, this
> > increases the chances for accessing the project, inspect, study,
> > etc.
>
> Exactly zero people have reported being unable to access the project
> because of xz. This is really getting silly.

You are right that the case of coreutils is not one of access, just of 
(reduced) safety in the access.

I would say that the lack of integrity checking in lzma-alone was a bad 
thing, but at least it was a known fact. With xz the situation is kind 
of deceptive. The user is induced to think that xz is safe (it even 
provides SHA256!), but depending on how the file was created and what 
decompressor is available, the integrity check in xz is sometimes 
performed and sometimes not. In particular, the case of decompressing 
coreutils tarballs with busybox is comparable to using lzma-alone.

But other projects may have in fact access problems. As Lasse Collin 
said[1], "XZ Embedded is also very limited. It cannot decompress all .xz 
files".

[1] http://lkml.iu.edu/hypermail/linux/kernel/1002.1/02383.html

At least some projects using optional xz features have already suffered 
(subtle) problems[2]:

[2] http://lkml.iu.edu/hypermail/linux/kernel/1403.1/02085.html
"But speaking as the Squashfs author, the lack of BCJ support for an 
architecture creates a subtle failure mode in Squashfs, this is because 
not all blocks in a Squashfs filesystem get compressed with a BCJ 
filter. At compression time each block is compressed without any BCJ 
filter, and then with the BCJ filter(s) selected on the command line, 
and the best compression for *that* block is chosen. What this means is 
kernels without a particular BCJ filter can still read the Squashfs 
metadata (mount, ls etc.) and read many of the files, it is only some 
files that mysteriously fail with decompression error. As such this will 
be (and has been) invariably treated as a bug in Squashfs."

One advantage of lzip (or gzip, or bzip2) over xz is that even the tiny 
educational decompressor lzd can decompress and check the integrity of 
even the largest files created with the mighty plzip on the largest 
computers in the world. From the point of view of the typical user, the 
lzip format just works and is safe, no strings attached.


Best regards,
Antonio.