Re: [Qemu-devel] [RFC 0/5] Slow-path for atomic instruction translation

[Alex Bennée]

Alvise Rigo <address@hidden> writes:

> This patch series provides an infrastructure for atomic
> instruction implementation in QEMU, paving the way for TCG multi-threading.
> The adopted design does not rely on host atomic
> instructions and is intended to propose a 'legacy' solution for
> translating guest atomic instructions.

Thanks for posting this. 

> The underlying idea is to provide new TCG instructions that guarantee
> atomicity to some memory accesses or in general a way to define memory
> transactions. More specifically, a new pair of TCG instructions are
> implemented, qemu_ldlink_i32 and qemu_stcond_i32, that behave as
> LoadLink and StoreConditional primitives (only 32 bit variant
> implemented).  In order to achieve this, a new bitmap is added to the
> ram_list structure (always unique) which flags all memory pages that
> could not be accessed directly through the fast-path, due to previous
> exclusive operations. This new bitmap is coupled with a new TLB flag
> which forces the slow-path exectuion. All stores which take place
> between an LL/SC operation by other vCPUs in the same memory page, will
> fail the subsequent StoreConditional.

Do you have any figures for contention with these page aligned exclusive
locks? On ARMv8 the global monitor reservation granule is IMDEF but
generally smaller than the page size. If a large number of exclusively
accessed variables end up in the same page there could be a lot of
failed exclusive ops compare to the real world.

>
> In theory, the provided implementation of TCG LoadLink/StoreConditional
> can be used to properly handle atomic instructions on any architecture.
>
> The new slow-path is implemented such that:
> - the LoadLink behaves as a normal load slow-path, except for cleaning
>   the dirty flag in the bitmap. The TLB entries created from now on will
>   force the slow-path. To ensure it, we flush the TLB cache for the
>   other vCPUs
> - the StoreConditional behaves as a normal store slow-path, except for
>   checking the state of the dirty bitmap and returning 0 or 1 whether or
>   not the StoreConditional succeeded (0 when no vCPU has touched the
>   same memory in the mean time).
>
> All those write accesses that are forced to follow the 'legacy'
> slow-path will set the accessed memory page to dirty.
>
> In this series only the ARM ldrex/strex instructions are implemented.
> The code was tested with bare-metal test cases and with Linux, using
> upstream QEMU.

Have you developed any specific test cases to exercise the logic?

>
> This work has been sponsored by Huawei Technologies Dusseldorf GmbH.
>
> Alvise Rigo (5):
>   exec: Add new exclusive bitmap to ram_list
>   Add new TLB_EXCL flag
>   softmmu: Add helpers for a new slow-path
>   tcg-op: create new TCG qemu_ldlink and qemu_stcond instructions
>   target-arm: translate: implement qemu_ldlink and qemu_stcond ops
>
>  cputlb.c                |  11 ++-
>  include/exec/cpu-all.h  |   1 +
>  include/exec/cpu-defs.h |   2 +
>  include/exec/memory.h   |   3 +-
>  include/exec/ram_addr.h |  19 +++-
>  softmmu_llsc_template.h | 233 
> ++++++++++++++++++++++++++++++++++++++++++++++++
>  softmmu_template.h      |  52 ++++++++++-
>  target-arm/translate.c  |  94 ++++++++++++++++++-
>  tcg/arm/tcg-target.c    | 105 ++++++++++++++++------
>  tcg/tcg-be-ldst.h       |   2 +
>  tcg/tcg-op.c            |  20 +++++
>  tcg/tcg-op.h            |   3 +
>  tcg/tcg-opc.h           |   4 +
>  tcg/tcg.c               |   2 +
>  tcg/tcg.h               |  20 +++++
>  15 files changed, 538 insertions(+), 33 deletions(-)
>  create mode 100644 softmmu_llsc_template.h

-- 
Alex Bennée