Re: [RFC 0/3] acpi: cphp: add CPHP_GET_CPU_ID_CMD command to cpu hotplug MMIO interface

[Laszlo Ersek]

On 10/10/19 20:15, Michael S. Tsirkin wrote:
> On Thu, Oct 10, 2019 at 05:57:54PM +0200, Igor Mammedov wrote:
>>> Then we should consider switching acpi to use fw cfg.
>>> Or build another interface that can scale.
>>
>> Could be an option, it would be a pain to write a driver in AML for fwcfg 
>> access though
>> (I've looked at possibility to access fwcfg from AML about a year ago and 
>> gave up.
>> I'm definitely not volunteering for the second attempt and can't even give 
>> an estimate
>> it it's viable approach).
>>
>> But what scaling issue you are talking about, exactly?
> 
> Just this: each new thing we add is an ad-hoc data structure with
> manually maintained backwards compatibility and no built-in discovery.
> 
> fw cfg has built-in discovery and we've finally managed to
> handle compatibility reasonably well.
> 
> PV is already very problematic.  Spreading PV code all over the place
> like this is a very bad idea.  For you CPU hotplug is something that you
> keep in mind first of all, but someone bringing up a new platform
> already has a steep hill to climb.  Adding tons of custom firmware is
> not helping things.
> 

- Custom firmware is unfortunately unavoidable in this case. SMM is a
privilege barrier between the firmware and the OS. The firmware needs to
care because to OS could use a hotplugged CPU to attack SMM, unless
hardware and firmware co-operate to prevent that. This whole ordeal aims
to prevent such an attack.

If you remove SMM from the picture (build OVMF without it), there is no
such privilege barrier between fw and OS, the firmware doesn't need to
know anything about VCPU hotplug events, and hotplug already works.


- Custom hardware is also expected. On physical platforms, there is a
dedicated BMC / RAS thingy that e.g. provides the APIC-ID to firmware,
when a CPU is hotplugged. One factor that makes this feature difficult
is that edk2 does not contain any code driving such hardware; it only
contains firmware code for *consuming* what the BMC / RAS thingy
produces. We're trying to work backwards from that. We could implement a
brand new hotplug controller device model, but it would be entirely PV
(= it would be our own design). Reusing the CPU hotplug register block
isn't worse, in that regard. This is my understanding anyway.


- Using fw_cfg *DMA* in an SMI handler (at OS runtime) could be
problematic. Especially if you consider SEV guests. For DMA transfers,
memory has to be decrypted and encrypted, which involves page table
manipulation.

In the firmware, this is handled with a SEV-specific
EDKII_IOMMU_PROTOCOL implementation. The fw_cfg library (transparently
to the caller) utilizes the IOMMU protocol, for dealing with DMA in SEV
guests.

Such protocols are boot-time only; they are not runtime protocols, let
alone SMM protocols. They are not available to SMI handlers.

Raw IO ports are extremely attractive in this regard: they don't depend
on page tables (SMM has its own set of page tables), they just work
under SEV out of the box. (The only exception is [REP] INS*/OUTS*, as
those depend on memory, not just registers, but no such instructions are
expected for the hotplug register block.)

- IO-port based fw_cfg might be suitable for the hotplug SMI handler,
yes. Assuming it's really only the firmware that needs information from
QEMU in the hotplug SMI handler, and not the other way around. (IO-port
based fw_cfg doesn't support writes.)

Now, when considering IO-port based fw_cfg in the SMI handler, we should
at least mention the risk of interfering with the Linux guest's own
fw_cfg driver. I've *never* been happy with anything non-firmware
accessing fw_cfg (it says "*firmware* config" in the name!), so I
wouldn't care much in practice. It's just that we should be aware that
there is a chance for entanglement. (The SMI handler cannot restore the
previous fw_cfg state, when it finishes up.)

At least on a theoretical level, the ACPI CPU hotplug register block is
nicer in this regard: the OS doesn't know about it at all (it is
accessed only through QEMU-generated AML code), and the plan is for the
same AML code to closely coordinate with the firmware. Namely, the AML
would raise the hotplug SMI.

Thanks
Laszlo