Re: [QEMU][ARM] Powering on remaining cores, bug?

[Peter Maydell]

On Fri, 11 Sep 2020 at 17:16, Sebastian Mueller
<sebastian.mueller2@yahoo.com> wrote:
> I'm passing my guest image to QEMU with -kernel. As far as I understand it 
> QEMU should perform the actions that a bit of firmware does on real hardware.
>
> qemu-system-aarch64 -machine virt -cpu cortex-a57 -machine virtualization=on 
> -machine secure=on -kernel aarch64.elf -display none -serial stdio -smp 2 -s 
> -S
>
> Without the bold marked part, i. e. only EL1 emulation, everything works fine.

Ah, I think I see what's happening here. QEMU intends to
support two different use cases for the case of booting with
EL2 and EL3 emulation enabled:

(1) booting a Linux kernel, which will start in EL2 and use
QEMU's PSCI emulation. The boot code notices that we're doing
this and it puts the other CPUs in a state where the PSCI
"turn this CPU on" code will work (in particular it will
set the EL3 registers for the secondary cores to say "EL2
is 64-bit"). You're not using this case because you passed
an ELF file to -kernel, not an Image file.

(2) booting bare-metal guest code (eg UEFI) which is
going to provide PSCI itself. In this case QEMU's own
PSCI handling is disabled, and instead QEMU starts all
cores powered on at once and lets the guest firmware
identify them, put them in wait-until-ready pens, etc.
You're not using this case because you didn't pass QEMU
your guest code via -bios or -pflash.

The bug in QEMU is that the check for "is this guest
code going to be started in EL3 so we should disable
PSCI and start all cores" is not exactly the same as
the check for "is this a Linux kernel so we should set
the secondary core state up so the PSCI calls work right",
so you can fall into a non-working gap between these two
use cases.

We should probably fix that bug so that passing a non-Linux
kernel (ie an elf file) to -kernel has the same "disable
PSCI" behaviour as using -bios or -pflash. You can work
around it by using -bios/-pflash to load your guest code
instead of -kernel. (NB that those options will want a
raw binary file, not an ELF.) You'll also need to change
your guest code so that it copes with having all CPUs
start at once and doesn't try to use PSCI.

The underlying principle driving this is that there is
only one EL3, so either (a) the guest code gets to
run there or (b) QEMU's PSCI emulation gets to behave
in a way that simulates there being firmware running there,
but you can't do both at once. So your guest code can
use the QEMU PSCI support, or it can run at EL3, but
not both.

thanks
-- PMM