Re: [Qemu-devel] Booting kernels with PVHVM documentation?

[Liam Merwick]

On 06/03/2019 17:51, Alex Bennée wrote:
> Hi,
>
> I've been looking at using PVH as an alternative to a long bios boot
> sequence to boot some x86_64 test kernels for tests/tcg. I'm finding it
> hard to piece together all the bits but I naively thought it would just
> be a case of adding a few ELF NOTES to my boot.S with something like:
>
>            ELFNOTE(Xen, XEN_ELFNOTE_VIRT_BASE,      _ASM_PTR 0x100000)
>            ELFNOTE(Xen, XEN_ELFNOTE_ENTRY,          _ASM_PTR _start)
>            ELFNOTE(Xen, XEN_ELFNOTE_PHYS32_ENTRY,   _ASM_PTR 0)    /* entry == 
> virtbase */
>            ELFNOTE(Xen, XEN_ELFNOTE_PADDR_OFFSET,   _ASM_PTR 0)
>
>            .code64
>            .section .text
>            /* Kernel Entry Point */
>    .global _start
>    _start:
>            // Setup stack ASAP
>            movq $stack_end,%rsp
>
> However I'm running into lots of head scratching as the get_elf_note
> code seems to skip over the notes before failing:
>
>    ./qemu-system-x86_64 -monitor none -display none \
>      -chardev stdio,id=out -device isa-debugcon,chardev=out \
>      -device isa-debug-exit,iobase=0xf4,iosize=0x4 -kernel ./tests/hello
>    load_elf64: processing hdr:0 of type 1
>    load_elf64: processing hdr:1 of type 4
>    get_elf_note_type64: looking for type 18, first is 3
>    get_elf_note_type64: 4/20
>    get_elf_note_type64: offset is 36
>    get_elf_note_type64: note is 0
>    get_elf_note_type64: 0/123713
>    get_elf_note_type64: offset is 123728
>    load_elf64: processing hdr:2 of type 1685382481
>    qemu-system-x86_64: Error loading uncompressed kernel without PVH ELF Note



What does 'readelf -Wn ./tests/hello' or 'readelf -p .notes 
./tests/hello' show?



> So I thought I'd go back to the Linux kernel and see if I could get it
> to boot up. So I built an x86_64 kernel with:
>
>    CONFIG_XEN_PVHVM=y
>    CONFIG_XEN_PVHVM_SMP=y
>    CONFIG_XEN_PVH=y
>    CONFIG_PVH=y
>
> And tried to boot that, it certainly gets a lot further but in detecting
> the note 18 it's looking for but then doesn't provide any output. So I
> started digging around the patches and saw talk of a PVH option ROM
> which does all the x86 mode escalation before booting the kernel.
> However I was unable to find any documentation about if I should be
> adding this manually to my command line or if it is auto-magiced into
> place. So I have a number of questions:
>
>    * what's the canonical command line for booting a Linux PVHVM kernel?

I had been using '-bios ./qboot.bin' (using 
https://github.com/bonzini/qboot/pull/17/files) but Stefano's changes 
mean that isn't necessary. Here's a command line I used

sudo $QEMU \
      -name testvm01 \
      -machine 
q35,accel=kvm,kernel_irqchip,nvdimm,nosmm,nosmbus,nosata,nopit \
      -cpu host \
      -m 1024,maxmem=20G,slots=2 \
      -smp 1 \
      -no-user-config \
      -nodefaults \
      -object 
memory-backend-file,id=mem0,share,mem-path=$IMAGE,size=235929600 \
      -device nvdimm,memdev=mem0,id=nv0 \
      -append 'console=ttyS0,115200,8n1 root=/dev/pmem0p1 panic=1 rw 
tsc=reliable no_timer_check noreplace-smp init=/usr/lib/systemd/systemd 
rootfstype=ext4 rcupdate.rcu_expedited=1 reboot=f' \
      -no-reboot \
      -serial mon:stdio \
      -kernel $KERNEL


>    * should this work in TCG as well?
>    * are they any special linker rules required for the Xen.notes?
>
> And finally:
>
>    * is this idea of mine a weird abuse of the PVHVM boot protocol or
>      does it make sense?


Regards,
Liam