Re: [Qemu-devel] [PATCH v4 6/7] Make the kernel image in the fw_cfg DMA interface bootable

[Laszlo Ersek]

On 10/02/15 14:07, Gerd Hoffmann wrote:
>   Hi,
> 
>>> Any -kernel boot on x86 will use either linuxboot.bin or multiboot.bin.
>>
>> (Except when your firmware is OVMF -- OVMF has its own LoadLinuxLib. So,
>> if you decide to extend linuxboot.bin / multiboot.bin with the DMA
>> capability, that can't regress OVMF by definition, and you certainly
>> won't hear me complain.)
> 
> What does ovmf expect btw?  linux kernel with efi stub I assume?

That's a hard question for me to answer :) (The library was written /
ported by Jordan, so I'm not responding from personal memory.)

In Dec 2014 - Jan 2015, Matt, Paolo, Jordan & myself had a long
discussion about the different ways to boot an EFI kernel (subject "the
different ways to boot an EFI kernel"). Ultimately Matt wrote an article:

http://www.uefidk.com/blog/linux-efi-boot-stub

To distill the discussion (and I hope Matt will correct me if I'm wrong,
although I'll be heavily stealing from his emails), there are three ways:

(1) Legacy EFI boot where the boot loader does *everything*
(2) EFI boot stub
(3) EFI handover protocol

In (1), the boot loader calls ExitBootServices().

"OvmfPkg/Library/LoadLinuxLib" supports this method, for directly
booting a kernel from fw_cfg, on the path that it calls "Old kernels
without EFI handover protocol".

The problem with (1) is that "all the smarts
of booting a Linux kernel on EFI platforms are in the boot loader",
which includes workarounds for platform bugs, and lock-step development
between kernel and boot loader.

(2) From Matt: "This method makes the Linux kernel appear to be
a PE/COFF executable. This allows us to perform bug workarounds early in
the kernel source because it's the kernel that calls ExitBootServices().
Additional goodness is obtained by the fact that you no longer need to
try as hard to keep kernel and boot loader development in sync because
the boot loader does very little and all smarts are in the kernel."

The general drawback here is that without a boot loader, you can "only"
load the kernel image (and the EFI stub of the kernel can "only" load
the initrd) from filesystems for which the firmware has support (ie.
those that it exposes with EFI_SIMPLE_FILE_SYSTEM_PROTOCOL interfaces).

OvmfPkg/Library/LoadLinuxLib does *not* support this method.

However, ArmVirtPkg/Library/PlatformIntelBdsLib/QemuKernel.c does. (In
fact, for AAVMF, this is the only method supported.) The "drawback"
remark about filesystems does not apply, because the kernel, initrd and
cmdline blobs are retrieved from fw_cfg, and exposed in a synthetic
(memory-only, read-only) EFI_SIMPLE_FILE_SYSTEM_PROTOCOL. Then we launch
the kernel with gBS->LoadImage() and gBS->StartImage(), and the kernel
loads the initrd with the standard EFI_SIMPLE_FILE_SYSTEM_PROTOCOL
member functions.

(3) From Matt again: 'The third option, the EFI handover protocol, is a
happy medium between the first two approaches. You split the "EFI boot
smarts" between the boot loader and EFI boot stub, which allows you to
a) load files from non-FAT file systems via the boot loader and b) leave
all the EFI bug workarounds to the kernel developers because the kernel
is still responsible for calling ExitBootServices().'

Later Matt also mentioned that under (3) you can have extra bells and
whistles (graphics etc) in your boot loader, while the kernel still gets
early firmware access.

OVMF supports this method too.

I'll add that the "FAT file system" restriction that (3) is supposed to
remedy is a bit laxer in general; even without a GRUB-like boot loader,
you can load a kernel from PXE / TFTP, and generally from anything that
looks like an EFI_SIMPLE_FILE_SYSTEM_PROTOCOL.

So the summary is this:
- OVMF can load "legacy" kernels, implementing an appropriate boot
  loader and calling ExitBootServices() (1)

- OVMF can load EFI stubbed kernels, implementing an absolutely minimal
  boot loader (3)

- AAVMF can load EFI stubbed kernels only (see the rationale in
  <https://github.com/tianocore/edk2/commit/23d04b58>), in method (2).
  This was written by me, and I chose this over (1) and (3) because:

  - (1) would have never worked / made any sense for aarch64 -- see
    again the commit msg I referenced,

  - (3) would have been overkill -- all the extras that could have been
    granted by an external boot loader were useless here, and the EFI
    stub dependency of (3) enabled the simpler, direct LoadImage() /
    StartImage() method.

... Paolo asked why OVMF hadn't opted for (2) as well, to which Matt
replied -- if I understand correctly -- that in parallel with the
kernel's facilities being developed for (3), OVMF was supposed to
support / exercise those facilities too.

Another question was if (2) could be enabled in / ported to OVMF -- it
could be, yes, but I'm hard pressed for any reason.

> Could
> you also load efi apps, i.e. something like "qemu -kernel shell.efi"?

I seem to remember that this has been suggested by Jordan as well. The
synthetic file system + LoadImage() + StartImage(), seen in (2), would
be theoretically suitable for this. However, at least three things
aren't a good match now:

- The way the synthetic filesystem is currently populated. One thing we
  need to put in there is the kernel image, to be launched from the
  firmware with LoadImage() + StartImage(). However, the kernel uses
  the filesystem too (it looks for the initrd file there), therefore we
  populate the fs with that file too. Such a filesystem may not a good
  match for other EFI executables (especially not if they expect a
  writeable working directory).

- If you boot the kernel successfully, then StartImage() never returns.
  If it returns, then that's mostly considered an error, and the usual
  boot option processing commences, as if you had never specified
  "-kernel". This is probably not appropriate when your payload was the
  UEFI shell: exiting the shell (any shell) is completely normal for a
  user, and starting to process boot options right after that is
  probably unexpected / unintended.

- When the top level kernel boot function returns (due to a genuine
  kernel boot error, or because "-kernel" wasn't specified,
  or -- fictively -- because the shell exited normally), the synthetic
  filesystem is torn down. This is okay for a kernel payload (because if
  booting it failed for the first time, there's no reason to retry it),
  but a user might want to reenter (and again exit) the shell any
  number of times.

... I think I prefer to keep the shell built into the firmware, and/or
to keep it on "UefiShell.iso". I already consider "-kernel" an abuse of
fw_cfg (one that at this point we can't get rid of any more); let's not
make it worse. I think the *real* goal here is an easy-to-use,
zero-config semihosting solution; ie. accessing a host directory tree
within the guest.

However, that use case is a burning problem for full-blown guest OS-es
as well. Assuming we end up with a simple solution, I think I'd prefer
to implement a UEFI_DRIVER that exposes the same host-side tree as an
EFI_SIMPLE_FILE_SYSTEM_PROTOCOL instance to the firmware, over
bastardizing fw_cfg even more.

I don't yet know how dynamic host-side writes should be shown to guest
firmware, but in the worst case, I could just return EFI_MEDIA_CHANGED:

    If the medium is changed while there are open file handles to the
    volume, all file handles to the volume will return
    EFI_MEDIA_CHANGED. To access the files on the new medium, the
    volume must be reopened with OpenVolume().

Thanks
Laszlo