Re: QEMU-KVM offers OPAL firmware interface? OpenBSD guest support?

[Fabiano Rosas]

David Gibson <david@gibson.dropbear.id.au> writes:

>> > * KVM PR doesn't currently work properly on POWER9, and getting it
>> >   working would be a significant amount of work
>> > * The way KVM PR works means it's very fiddly to get right, so it's
>> >   unlikely to ever be suitable for production work
>> > * Depending on host and guest cpu models there might be a few corner
>> >   cases it can never get exactly right
>> 
>> Out of curiosity what are the problems with KVM-PR on POWER9 currently and
>
> I don't know entirely.  My point is that I've never had the time
> and/or interest to investigate (and to my knowledge no-one else has
> either).  The biggest is likely to be that PR also needs to emulate to
> some extent the guest CPU's MMU.  That means adding support for the
> POWER9 radix-MMU, which would be avery large job.  There are probably
> other problems as well: I vaguely recall that if you attempt to boot a
> kernel, the first problem you hit is a new-in-POWER9 privileged SPR
> which PR doesn't emulate.

Just as a data point, I recently ran KVM-PR in POWER9/HPT. Works both in
bare-metal and nested. For a very simple guest, at least.

qemu-system-ppc64 -accel kvm -display none -vga none -machine \
pseries,cap-ccf-assist=off,cap-large-decr=off,cap-fwnmi=off,kvm-type=PR,kernel-irqchip=off
 \
-serial mon:stdio -smp 2 -m 1024 -kernel ~/vmlinux-guest -initrd \
~/initramfs.img -append "console=hvc0 debug mitigations=off"

I needed kernel-irqchip=off to avoid an issue with XICS and
mitigations=off because it seems we don't emulate the mtspr TRIG2
instruction which is used for L1 cache flush in some mitigations.

About running PR in radix, the code has this comment:

static int kvmppc_core_check_processor_compat_pr(void)
{
        /*
         * PR KVM can work on POWER9 inside a guest partition
         * running in HPT mode.  It can't work if we are using
         * radix translation (because radix provides no way for
         * a process to have unique translations in quadrant 3).
         */
        if (cpu_has_feature(CPU_FTR_ARCH_300) && radix_enabled())
                return -EIO;
        return 0;
}

which might indicate it would not be at all possible to support PR in
Radix? I don't know enough about PR to understand why.

>> what are the corner cases that it can never get right?
>
> Well, I'm not certain they exist (at least in a way that can't be
> worked around) - but I'm not certain they don't either.  In particular
> behavioural differences in userspace (i.e. MSR[PR] == 1) between host
> and guest CPUs couldn't be handled by PR (since it wouldn't get the
> necessary trap to emulate).  Those are rare, of course, since the
> architecture is pretty strict about user mode behaviour, but I can't
> swear that none exist.  There are certainly new non-privileged
> instructions that have been added, but it might be possible to work
> around those (trap the illegal instruction and emulate if guest has it
> and host doesn't, hope the guest doesn't rely on the 0x700 trap if
> host has it and guest doesn't).
>
> Or of course you could have a big matrix of host/guest CPU userspace
> compatibility, but that in itself is not a trivial job.
>
>> This info may be
>> useful for those interested in fixing and using it and having it listed here
>> may save time debugging some known problems.
>> 
>> Regards,
>> BALATON Zoltan