Re: [Bug-xorriso] Creating a data partition after writing iso-hybrid image

[Thomas Schmitt]

Hi,

Andreas Heinlein wrote:
> We now want to include a tool within the live system that creates a
> persistence partition within the remaining space on the USB drive. We
> are however unsure how to do this, because of the non-standard MBR and
> GPT which xorriso creates.

Well, this is actually the ISOLINUX isohybrid layout.
xorriso is only the henchman in this crime.

There are several compliance problems with isohybrid's partition layout:
- The EFI System Partition is nested inside the ISO 9660 partition.
- There is a GPT although the MBR partition table does not constitute
  a "protective MBR" with a single partition.
- There might be an Apple Partition Map.

Each of them can confuse a partition editor.

This does not have to be so:
grub-mkrescue for example creates for BIOS+EFI an EFI compliant GPT
with protective MBR.
The wrapper script
  
https://dev.lovelyhq.com/libburnia/libisoburn/raw/master/frontend/grub-mkrescue-sed.sh
offers a mode "mbr_only" which when used as "xorriso" by grub-mkrescue
creates an MBR based EFI compliant partitioning. (Needs xorriso >= 1.4.4)

I personally deem EFI without GPT better suited for images (ISO or not),
because GPT refers to the end of the device which normally is not known
when the image gets created. The GPT backup block at the end of the image
would actually have to be relocated after the image was put onto a device.


> For testing, I could create a partition in the free space in MBR only
> using fdisk (gdisk refused to accept and alter the GPT), and it seems to
> work.  But I am courious if this will always work and if this is the
> right way to do?

I would consider to do the partitioning by own byte-level operations
rather than by a partition editor which can change its habits.
On the other hand, if your ISO provides the partition editor, then you
can rely on your testing that it works.


> Is there any guarantee that the linux kernel will
> always use the MBR table and ignore the GPT table?

The GPT in isohybrid is actually invalid and no EFI/GPT compliant software
should interpret it - in an ideal world. (The world flaw, why Matthew Garrett
included GPT in his isohybrid EFI layout at all, is not known to me, i fear.)

The Apple Partition Map can be avoided easily at ISO production time by
not giving xorrisofs option -isohybrid-apm-hfsplus .


In order to make an ISOLINUX isohybrid more digestible to partition
editors, i would first smash the GPT by zeroizing bytes 512 to 1023
of the ISO and also the backup GPT header block at the end of the ISO.

The 512-block address of the GPT backup header block can be inquired by e.g.

  xorriso -indev debian-8.4.0-amd64-netinst.iso \
          -report_system_area plain \
  2>&1 | grep 'GPT lba range' | awk '{print $7}'

which picks the third number from a line like
  GPT lba range      :      64  505802  505855

(For an explanation of -report_system_area headers, see
   xorriso -report_system_area help 2>&1 | less
)

This backup GPT header address may also be read directly from the ISO
as 64 bit little-endian number at byte 512 + 32 to 512 + 39.

If both 512-blocks, 1 and 505855 are zeroized, then there is no entry point
for any reasonable GPT interpretation. So a partition editor can only see
MBR and possibly APM if present.


Not so easy to fix is the nesting of the two MBR partitions.

You'd have to reduce the size of partition 1 so that it only reaches up
to the start of partition 2. Then a new partition 3 would have to span
from the end of partition 2 to the end of the ISO.

This will prevent proper mountability of the ISO by a partition. So you
have to weigh what is more important: partiton sanity or udev and mount
sanity.
(Last time i looked, udev attributed the properties of partition 1 to
the base device, too.)

Mainly tricky with MBR partitions on byte level is the cylinder-head-sector
address for which one would have to guess the sizes of cylinder and head.
Afaik, fdisk guesses quite poorly.
xorriso tries to do better with its command -report_system_area:
  MBR heads per cyl  : 64
  MBR secs per head  : 32

------------------------------------------------------------------------

It is possible to resolve the problem of nested partitions at ISO production
time by appending the EFI boot image as partition after the ISO.

Within the xorriso wrapper for grub-mkrescue the GPT-less BIOS+EFI layout
"mbr_only" is achieved by these xorrisofs options:
  
  -no-pad -append_partition 2 0xef /tmp/$efi_tmp_name \
  ... BIOS boot options -b et al ... \
  -eltorito-alt-boot \
  -e --interval:appended_partition_2:all:: -no-emul-boot \
  --protective-msdos-label

Option --protective-msdos-label may be omitted to get a mountable ISO
partition. (Else the partition around the ISO will start at 512-block 1
instead of block 0.)
Vladimir Serbinenko of GRUB prefers partition start at block 1, but 
start block 0 works fine too.

The strange -e path points to the appended partition 2, which holds
the EFI System Partition FAT filesystem. /tmp/$efi_tmp_name is the path
to the FAT image prepared by grub-mkrescue on hard disk.

This way the EFI System Partition is outside of the ISO filesystem and
there is no space wasted by having an El Torito boot image file for EFI
inside the ISO.

One can do similar with older xorriso versions by having two copies of
the EFI boot image: one as file in the ISO for -e, and one appended 
by -append_partition with type 0xef.



> We basically followed Ubuntu instructions; the xorriso command we use is:
> xorriso -as mkisofs -r -V "UPR_12.04r1" -o $CURDIR/upr-12.04r1.iso \
>    -isohybrid-mbr /usr/lib/syslinux/isohdpfx.bin -partition_offset 16 
> -cache-inodes -J \
>    -l -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot 
> -boot-load-size 4 \
>    -boot-info-table -eltorito-alt-boot -e boot/grub/efi.img -no-emul-boot \
>    -isohybrid-gpt-basdat $CURDIR/extract-cd

That's a typical BIOS+EFI isohybrid setup (without the useless APM of Debian).

I will make experiments with a Debian amd64 ISO whether it is possible to
modify this command by the gestures of above "mbr_only".


Have a nice day :)

Thomas