Re: Large files truncated using xorriso 1.5.4 using zisofs v2

[Thomas Schmitt]

Hi,

Dag Wieers wrote:
> It appears that these files in the ISO report the compressed size (so less
> than 50%) rather than the real size, only when the file is larger than 4GB.
> [...]
>     -zisofs default:version_2=as_needed
> [...]
> On the mounted ISO:

zisofs2 is not yet supported by the Linux kernel. (And old zisofs does not
encode files larger than 4 GiB - 1 byte.)

You see indeed the compressed size.
In your system log, there should be a complaint like:
  isofs: Unknown ZF compression algorithm: PZ

This run should show the original uncompressed size:

  ISO=...path.to.image.or.optical.derive...

  xorriso -indev "$ISO" -lsl /iso/images/foobaz-1.2.3.tar --


This run will restore the uncompressed tarball to hard disk:

  xorriso -osirrox on -indev "$ISO" \
          -extract /iso/images/foobaz-1.2.3.tar ./foobaz-1.2.3.tar


But for more general accessability, consider to compress the tarball by
the --gzip filter:

  -zisofs default:version_2=off
  -find /images -type f -name *.tar -pending_data -exec set_filter --zisofs --
  -set_filter --gzip /iso/images/foobaz-1.2.3.tar --

or

  -find /images -type f -name *.tar -pending_data -exec set_filter --gzip --

This will add the suffix .gz to the file name.

(Possibly i should introduce a -find test for file size.)


--------------------------------------------------------------------------

The zisofs2 format was developed on initiative of Valentín KIVACHUK BURDÁ
and is documented in
  
https://dev.lovelyhq.com/libburnia/libisofs/raw/branch/master/doc/zisofs2_format.txt
I am not sure whether Valentín finished his planned decompression program
for the mounted files of an unaware Linux system.

I have a well tested kernel patch for Linux 5.9, which was 2 years ago.
Maybe it still applies, as the isofs module is not much moving.

But when i posted a fix for a flaw in the existing zisofs code of Linux:
  https://lore.kernel.org/linux-scsi/20201120140633.1673-1-scdbackup@gmx.net
  "[PATCH] isofs: fix Oops with zisofs and large PAGE_SIZE"
i got no replies.

A patch for improving throughput of zisofs decompression by avoiding to
read each 32 KB block 8 times died together with with the .readpages()
method of struct address_space_operations. I would have to adapt it to the
.readahead() method. But given that i can't get a reply to a report about
a kernel Oops() on powerpc64 machines, i lack of motivation.

The zisofs2 support in xorriso exists so that example ISOs can be made
and for backup use cases, where restoring only by xorriso is acceptable.

If you know somebody who would be interested in bringing zisofs2 into
the Linux kernel, please forward my mail address or ask me for the 2 year
old patch.


Have a nice day :)

Thomas