Re: [Bug-xorriso] xorriso and Fedora livecd-tools

[Thomas Schmitt]

Hi,

Neal Gompa wrote:
> If I could use hfsplus for ppc, what are the options I need? I'd like
> to have _some_ compatibility with PowerPC systems in place...

I had the hope that 32 bit PowerPC will die out finally. But when i discussed
the HFS problem with Helge Deller, Debian maintainer of several exotic
architectures, he stated that big-endian 64 bit PowerPC will again need HFS
although he could not outrule that HFS+ would be ok.

I proposed to him to then attempt what Matthew J. Garrett did for Fedora
and HFS+, namely to have a small HFS image as data file in the ISO and
to mark it by an Apple Partition Map entry.
That HFS image would get produced by external means.

If you really need to support old PowerPC and do not want to keep genisoimage
for that purpose, then consider to go that way. (One would have to analyse
what Apple specific files get into the ISO and how to treat them in respect
to the special HFS file attributes.)


When xorriso got its -hfsplus support for grub-mkrescue (as contribution
by Vladimir Serbinenko) it was tested for Debian "powerpc" and did not
work. (Of course this may have been because of our cluelessness about the
real needs of old PowerPC firmware. We only had a VM and a genisoimage
ISO as example.)

There is a "ppc64el" Debian architecure. The dedicated ISOs are not
equipped with HFS but rather comply to CHRP. From my cheat sheet
libisofs-*/doc/boot_sectors.txt :
-------------------------------------------------------------------------
Sources:
    Mail conversations with Vladimir Serbinenko.
    http://stuff.mit.edu/afs/sipb/contrib/doc/specs/protocol/chrp/chrp1_7a.pdf
    
https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/W51a7ffcf4dfd_4b40_9d82_446ebc23c550/page/PowerLinux%20Boot%20howto

CHRP is marked by an MBR partition entry of type 0x96 spanning the whole
ISO 9660 image.

The specs in chrp1_7a.pdf promise that CHRP also recognizes ISO 9660 file
systems on unpartitioned disks. (See 11.1.1. Media Layout Format)

The firmware looks up a file /ppc/bootinfo.txt which in SGML-ish tag
<boot-script> contains firmware commands.
E.g. to execute the binary /boot/grub/powerpc.elf as first stage of GRUB2:
  <boot-script>boot &device;:\boot\grub\powerpc.elf</boot-script>
-------------------------------------------------------------------------

If there were any PowerPC systems in reach, one could make experiments.


===========================================================================

> >> large images that require UDF

> > How that ? Do they need to read files of 4 GiB or more on Solaris or
> > one of the BSDs ?

> CentOS and a few other downstream derivatives of Fedora produce huge
> ISOs at ~8GB.

Even the handicapped systems have no problem with the overall size of ISOs.
The problems are about:

- Data files of size 4 GiB or larger. In ISO 9660 they get represented
  by more than one consequtive directory records with identical names.
  Solaris and BSD are not aware of this rule. See e.g.:
    https://mail-index.netbsd.org/netbsd-bugs/2014/07/02/msg037276.html

- Directory records (combined dirents + inodes) stored above 4 GiB.
  In BSD, directory records are addressed by their inode number which is
  simply the byte address of the record. Since BSD inode numbers once were
  only 32 bit, many versions still roll over this byte address if the
  directory record is stored above 4 GiB.
  This only affects multi-session ISOs. Not large single session ISOs
  which have their directory tree at very low byte addresses.
  Some BSDs with 64 bit inode numbers can work around meanwhile:
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=190655


> http://ftp.osuosl.org/pub/centos/7/isos/x86_64/CentOS-7-x86_64-Everything-1708.iso

Can't we have small examples of large ISOs ? :))

> My understanding is that UDF support is required for these.

Not on Linux kernels, not on MS-Windows.
And also not if no large data files are in the ISO.

... still downloading the Everything ISO ... at 5 MB/s ... whew, done.

  $ ls -lR /mnt/iso | sort -n -r -k 5 | head -3
  -rw-r--r-- 1 root root 368574464 Sep  5  2017 squashfs.img
  -rw-rw-r-- 1 root root 134349968 Nov 20  2016 qt-doc-4.8.5-13.el7.noarch.rpm
  -rw-rw-r-- 1 root root 118650832 Jul  4  2014 
kdeartwork-wallpapers-4.10.5-4.el7.noarch.rpm

So the biggest data file has 368,574,464 bytes. Far away from the 4 GiB
limit for ISO 9660 drivers which still bear the old names "High Sierra"
or "CDFS".

(Elsewise "Everything" is a mjg-style isohybrid with invalid GPT but no APM.
 Much like the debian-cd amd64 ISOs. Should be well doable with xorriso.)


If only Linux mountability is of concern, one could already now make tests
with explicit mount -t "iso9660" to check whether the files in the ISO
then match their originals on hard disk.


===========================================================================

> > the GPT of Matthew J. Garret's layout is simply invalid
> > Any specs compliant EFI will use the MBR partition of type 0xef and ignore
> > the EFI partition entry in the GPT.

> Would that mean that EFI systems would not prefer UEFI boot then?

No. It just means that EFI firmware follows the MBR partition table item of
type 0xef and gets to the EFI System Partition.

In Fedora-Live-Xfce-x86_64-rawhide-20150704.iso there are several pointers
to the EFI partition:

  $ xorriso -indev Fedora-Live-Xfce-x86_64-rawhide-20150704.iso \
            -report_el_torito plain -report_system_area plain
  ...
  El Torito images   :   N  Pltf  B   Emul  Ld_seg  Hdpt  Ldsiz         LBA
  ...
  El Torito boot img :   2  UEFI  y   none  0x0000  0x00  10136          43
  ...
  El Torito img path :   2  /isolinux/efiboot.img
  ...
  MBR partition table:   N Status  Type        Start       Blocks
  ...
  MBR partition      :   2   0x00  0xef          172        10136
  ...
  MBR partition path :   2  /isolinux/efiboot.img
  ...
  GPT partition name :   2  490053004f00480079006200720069006400
  GPT partname local :   2  ISOHybrid
  GPT partition GUID :   2  524fd0a25d2862489fa91dda49601a02
  GPT type GUID      :   2  a2a0d0ebe5b9334487c068b6b72699c7
  GPT partition flags:   2  0x0000000000000000
  GPT start and size :   2  172  10136
  GPT partition path :   2  /isolinux/efiboot.img
  ...
  APM partition name :   1  EFI
  APM partition type :   1  Apple_HFS
  APM start and size :   1  43  2534
  APM partition path :   1  /isolinux/efiboot.img

These four pointers are all directed to the same block range (sometimes
given in blocks of 512, sometimes of 2048) which is the range of ISO file
/isolinux/efiboot.img.
In there is the usual FAT filesystem with EFI start programs like
  /EFI/BOOT/BOOTX64.EFI
and possibly some second stage programs or supplementary data files.

No specs compliant EFI is supposed to consider the GPT (because invalid)
or the APM (because not mentioned in specs). If the ISO is found on DVD,
then EFI will follow the El Torito Catalog to Boot Image number 2.
As said, on USB stick or hard disk it will use MBR partition 2.


The APM is for some Macs which according to Matthew J. Garrett pretend
to be EFI but still want to see APM and HFS. It is not so clear why in
his isohybrid.c changeset he puts the EFI image into APM as HFS, although
only APM partition number 2 bears an HFS:
  APM partition name :   2  EFI
  APM partition type :   2  Apple_HFS
  APM start and size :   2  2597  10144
  APM partition path :   2  /isolinux/macboot.img

===========================================================================


Have a nice day :)

Thomas