Re: [Qemu-devel] ARM QEMU/KVM and TrustZone

[Christoffer Dall]

On Sat, Jun 16, 2012 at 9:09 PM, Peter Maydell <address@hidden> wrote:
> On 16 June 2012 18:37, Christoffer Dall <address@hidden> wrote:
>>> On 22 May 2012 13:22, Peter Maydell <address@hidden> wrote:
>>>> Historically for QEMU we haven't implemented TrustZone support even
>>>> though we claim to emulate CPUs that provide it. Instead we provide a
>>>> CPU which mostly looks like a variant of the real thing without the
>>>> TrustZone feature. We then bolt on a few extra cp15 registers (eg the
>>>> SCR) as a pragmatic move to get Linux guests to run. Now we're also
>>>> dealing with KVM on ARM I'd like to define things a bit more solidly
>>>> so KVM and TCG agree on what the CPU model they present is.
>>>>
>>>> There are several possible environments we could provide
>>>> to a guest:
>>>>  (1) a CPU with full TrustZone support
>>>>  (2) a CPU without TrustZone at all
>>>>  (3) a TZ CPU running in NonSecure PL0/PL1
>>>>  (4) a TZ CPU running in Secure PL0/PL1
>>>>
>>>> In some ways (1) is the "purist" solution -- emulate exactly what the
>>>> hardware does. However:
>>>>
>>>>  * on TCG it would require a lot of work, including new functionality
>>>>   in core QEMU (to support having different CPU cores being able to
>>>>   see different views of memory, and having the S/NS attribute
>>>>   attached to memory transactions)
>>>>
>>>>  * it isn't possible in KVM, because the ARM Virtualization Extensions
>>>>   don't allow you to fake the CPSR a guest sees, and so you can't
>>>>   make the guest believe it is in Monitor mode
>>>>
>>>> Option (2) is architecturally sanctioned (ie TrustZone is an optional
>>>> feature, not mandatory), but it doesn't correspond to real CPUs, in
>>>> that the hardware Cortex-A8/A9/A15 always have TrustZone. So we're
>>>> modelling something that doesn't really exist.
>>>>
>>>> Options (3) and (4) correspond to the environment an OS guest
>>>> typically actually uses on hardware. For ARM's devboards (versatile
>>>> express etc) Linux runs in the Secure world but it doesn't actually
>>>> use any of the TrustZone functionality, it's just a "give me full
>>>> access to everything" setup. For just about every other ARM system,
>>>> the boot rom or equivalent keeps Secure world to itself, and the OS
>>>> kernel runs in the NonSecure world. (This typically means that the
>>>> boot rom provides a set of board-specific entry points via the Secure
>>>> Monitor Call (SMC) instruction for doing operations like "invalidate
>>>> whole L2 cache" which require secure privileges.)
>>>>
>>
>> Is there anything preventing people from writing a small bootloader
>> that switches into non-secure mode and runs kernels there as a general
>> approach (apart from laziness)?
>
> I'm not sure what you're suggesting here. Mostly kernels do run
> in NS mode on hardware, except for on ARM devboards. On ARM
> devboards if you try to run the kernel in NS mode it will fall
> over the first time it tries something that needs secure world
> privileges. For KVM my rule of thumb is that it needs to run
> the same kernel the hw runs.

I was simply under the assumption that all operations required for a
kernel to run was allowed in NS-mode, but if that's not the case, then
never mind.

>
> To some extent the small boot loader would be the thing I describe
> as a 'fake bootrom' below.
>
>>>> Proposal:
>>>>
>>>> My suggestion is that we present the guest with a view that looks like
>>>> a sort of superset of (2) (3) and (4), ie sufficient that a guest
>>>> expecting any of those environments can run. In particular:
>>>>
>>>>  * no cp15 registers have secure/nonsecure banking
>>>>  * there is only one memory space visible
>>>>  * secure-access-only permissions are not enforced
>>>>  * the handful of only-in-trustzone registers are implemented
>>>>   (eg VBAR, MVBAR)
>>>>  * we implement a "fake monitor mode"
>>>>
>>>> The aim of the "fake monitor mode" is to allow us to provide fake
>>>> qemu-specific bootroms which implement whatever the board's SMC
>>>> interface is, without having to write specific KVM kernel code for
>>>> each board. So we don't have to run arbitrary secure-world guest code.
>>>> The rules are:
>>>>  * on an SMC instruction we enter the guest at the SMC vector
>>>>   as defined by the MVBAR (monitor vector base address register)
>>>>  * we actually run with the same access permissions as above
>>>>   (and under KVM if you look at CPSR.M it will tell you you're
>>>>   in Supervisor mode)
>>>>  * return from the SMC is via a standard exception return insn
>>>>  * we don't implement the separate memory space for the secure
>>>>   world. (This implies that you need to find space in the
>>>>   non-secure world's physical memory map for the bootrom shim;
>>>>   not a big deal I think since we already have a requirement
>>>>   for some space to put QEMU's arm_boot trivial bootloader.)
>>>>
>>
>> you could have a separate set of stage-2 translation tables for this
>> and keep things separate for real (or would we rely on fake-SMC code
>> to directly be able to read fake-non-secure data, which is still
>> possible through a different memory map I guess).
>
> I'm trying to keep things simple (and separate memory maps for TCG
> is a bit tricky). We can't make the monitor mode really look like
> monitor mode, so there's not much point trying to implement all
> the difficult complicated bits when we control the code running in
> this mode anyway.
>

ok, fair enough.

>> what kind of operations would be required from SMC calls in a KVM
>> guest setting? I can see this in an embedded market, but are they not
>> likely to even capture Hyp mode already and set things up as required?
>> What I mean is, if KVM is currently targeting Calxeda-type setups will
>> we ever run kernels that require SMC operations as guests?
>
> Yes, I think we must, because the CPU hardware doesn't let kernels
> do everything in NS mode. There are usually only a handful of
> operations needed. I've given some A8 examples above; I'm afraid
> I don't have time to check for the A15 equivalents (I have a plane
> to catch later :-)).
>
>> It feels a bit premature to implement all this.
>
> Basically it works at the moment for the vexpress A15 guest
> because it happens to be one of the special cases which runs
> in secure mode, and our cp15 emulation can just include
> enough rope to let it all work. However that is likely to
> drift into the ill-defined area where we are running the
> guest in something that's actually NS but the cp15 emulation
> lets it appear to access some S mode only registers.
>
> In any case, I already have this problem in TCG mode for the
> OMAP3 emulation, and so I want to define how it should all
> work for that. We don't need to implement the KVM side just
> yet as long as we're confident that we could...
>
If we do it in one memory space it doesn't sound very intrusive for
KVM, so in that regards I'm fine with the approach if it cannot be
avoided all together.

-Christoffer