Re: [Qemu-devel] Help on TLB Flush

[Lluís Vilanova]

Mark Burton writes:

>> On 12 Feb 2015, at 16:38, Alexander Graf <address@hidden> wrote:
>> 
>> 
>> 
>> On 12.02.15 15:58, Peter Maydell wrote:
>>> On 12 February 2015 at 14:45, Alexander Graf <address@hidden> wrote:
>>>> almost nobody except x86 does global flushes
>>> 
>>> All ARM TLB maintenance operations have both "this CPU only"
>>> and "all TLBs in the Inner Shareable domain" [that's ARM-speak
>>> for "every CPU core in the cluster"] variants (the latter
>>> being the TLB *IS operations). Looking at Linux's
>>> arch/arm64/mm/tlb.S and arch/arm64/include/asm/tlbflush.h
>>> most of the operations defined there use the IS variants.
>> 
>> Wow, did anyone benchmark this? I know that PPC switched away from
>> global flushes and instead tracks the CPUs a task was running on to
>> limit the scope of CPUs that need to flush.

> Doesn’t that mean you have to signal a specific CPU to cause it to flush 
> itself…. Isn’t that in itself expensive? Do you have to organise some sort of 
> atomicity yourself around that too?

Yup. AFAIR, Linux in x86-64 queues a request to a per-CPU request list, and uses
IPIs to signal these types of operations to the target CPU:

  http://lxr.free-electrons.com/source/kernel/smp.c?v=2.6.32#L386

Waiting for completion is implemented on top by incrementing some counter from
each CPU, and waiting for it to have the correct final value.

If something were implemented on these lines, it could be used as a generic
cross-CPU event messaging infrastructure (plus some interrupt bit in the CPU
structure that TCG would check to break away from guest code; I believe
something similar is already being used - icount? -).

PS: To be honest, I still don't know which TLBs we're talking about here, and
    which cases trigger these TLB flush operations.


Cheers,
  Lluis

-- 
 "And it's much the same thing with knowledge, for whenever you learn
 something new, the whole world becomes that much richer."
 -- The Princess of Pure Reason, as told by Norton Juster in The Phantom
 Tollbooth