Re: [Qemu-devel] QEMU/NEMU boot time with several x86 firmwares

[Stefano Garzarella]

Hi Rob,

On Tue, Nov 27, 2018 at 10:57 AM Rob Bradford <address@hidden> wrote:
>
> Hi Stefano,
>
> On Mon, 2018-11-26 at 17:40 +0100, Stefano Garzarella wrote:
> > Hi Samuel, Rob,
> > I'm proceeding to compare several x86 firmwares in order to
> > understand
> > which suits better with -kernel option to have a fast boot with
> > QEMU/NEMU.
> >
> > For your use cases, what boot time do you expect?
> >
>
> Kata Containers, one of the use cases we're looking at, is very
> sensitive to boot time, although it is configurable, the default
> configuration is to use qemu-lite with it's direct to kernel loader.
>
> That is probably a baseline we should be comparing with.
>
> As an aside, yesterday I posted the Seabios patches for enabling on
> NEMU/virt:
>
> https://github.com/rbradford/seabios/tree/virt-x86
>
> I didn't yet start optimising the configuration file we use for it
> beyond getting it < 128KiB or apply any optimisation patches.
>
> And I also have some qboot patches too:
>
> https://github.com/rbradford/qboot/tree/virt-x86
>
> So we're in a situation where we can start comparing other machine
> types against virt too.

Thanks for the pointers, I'll try to do the same tests with qemu-lite
and direct kernel loader and also with NEMU "virt" machine, using your
versions of SeaBIOS and qboot.

>
> >
> > I compared SeaBIOS, qboot, and OVMF. I started each test using this
> > qemu parameters: "./qemu-system-x86_64 -bios path/to/bios.bin -m 1G
> > -cpu host -M accel=kvm -vga none -kernel path/to/bzImage -initrd
> > path/to/rootfs.cpio ..."
> >
> > As Samuel suggested, I added the total time to userspace adding a
> > probe in the kernel_init(), so the times (in msec) that I measured
> > are:
> > - qemu_init_end: first kvm_entry (i.e. QEMU initialized has finished)
> > - fw_start: first entry of the firmware
> > - fw_do_boot: after the firmware initialization (e.g. PCI setup,
> > etc.)
> > - linux_start_boot: before the jump to the Linux kernel
> > - linux_start_user: before starting the init process
> >
> > * SeaBIOS
> > Default configuration without debug messages (CONFIG_DEBUG_LEVEL=0) +
> > Stephen's patch (tpm: Check for TPM related ACPI tables before
> > attempting hw) + my patch (qemu: fast boot when linuxboot optionrom
> > is
> > used).
> >  qemu_init_end: 41.634812
> >  fw_start: 41.857374 (+0.222562)
> >  fw_do_boot: 52.754109 (+10.896735)
> >  linux_start_boot: 54.117220 (+1.363111)
> >  linux_start_user: 495.684199 (+441.566979)
> >
> > * qboot
> > Default configuration + my patch (pci: reduce pci_foreach() calls).
> >  qemu_init_end: 40.233717
> >  fw_start: 40.384048 (+0.150331)
> >  fw_do_boot: 45.660497 (+5.276449)
> >  linux_start_boot: 47.252119 (+1.591622)
> >  linux_start_user: 509.173886 (+461.921767)
> >
> > * OVMF (https://github.com/intel/ovmf-virt)
> > I followed this script
> > (
> > https://github.com/intel/nemu/blob/topic/virt-x86/tools/CI/run_nats.sh
> > )
> > to build OVMF.
> > Note: I put the "fw_start" probe in the BdsEntry()
> > [MdeModulePkg/Universal/BdsDxe/BdsEntry.c], I'm not sure if it is
> > "near" to the real entry point.
> >  qemu_init_end: 42.734555
> >  fw_start: 163.611506 (+120.876951)
> >  fw_do_boot: 369.713760 (+206.102254)
> >  linux_start_boot: 370.960364 (+1.246604)
> >  linux_start_user: 796.799667 (+425.839303)
> >
> > For OVMF case, are reasonable the times that I measured? Do you use a
> > different configuration?
> >
>
> That is the same OVMF configuration as we normally use, I have the
> configuration file checked in. Those numbers look like the ones I
> expect.

Perfect!

Please, let me know if you have something more important than boot
time to help you.

Regards,
Stefano

>
> > Thanks,
> > Stefano
> >
>
> Great work,
>
> Rob
>

--
Stefano Garzarella
Red Hat