[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-devel] [PULL v2 07/19] translate-all: make l1_map lockless
From: |
Richard Henderson |
Subject: |
[Qemu-devel] [PULL v2 07/19] translate-all: make l1_map lockless |
Date: |
Fri, 15 Jun 2018 09:43:42 -1000 |
From: "Emilio G. Cota" <address@hidden>
Groundwork for supporting parallel TCG generation.
We never remove entries from the radix tree, so we can use cmpxchg
to implement lockless insertions.
Reviewed-by: Richard Henderson <address@hidden>
Reviewed-by: Alex Bennée <address@hidden>
Signed-off-by: Emilio G. Cota <address@hidden>
Signed-off-by: Richard Henderson <address@hidden>
---
accel/tcg/translate-all.c | 24 ++++++++++++++----------
docs/devel/multi-thread-tcg.txt | 4 ++--
2 files changed, 16 insertions(+), 12 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 52e62125ed..52273e5846 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -469,20 +469,12 @@ static void page_init(void)
#endif
}
-/* If alloc=1:
- * Called with tb_lock held for system emulation.
- * Called with mmap_lock held for user-mode emulation.
- */
static PageDesc *page_find_alloc(tb_page_addr_t index, int alloc)
{
PageDesc *pd;
void **lp;
int i;
- if (alloc) {
- assert_memory_lock();
- }
-
/* Level 1. Always allocated. */
lp = l1_map + ((index >> v_l1_shift) & (v_l1_size - 1));
@@ -491,11 +483,17 @@ static PageDesc *page_find_alloc(tb_page_addr_t index,
int alloc)
void **p = atomic_rcu_read(lp);
if (p == NULL) {
+ void *existing;
+
if (!alloc) {
return NULL;
}
p = g_new0(void *, V_L2_SIZE);
- atomic_rcu_set(lp, p);
+ existing = atomic_cmpxchg(lp, NULL, p);
+ if (unlikely(existing)) {
+ g_free(p);
+ p = existing;
+ }
}
lp = p + ((index >> (i * V_L2_BITS)) & (V_L2_SIZE - 1));
@@ -503,11 +501,17 @@ static PageDesc *page_find_alloc(tb_page_addr_t index,
int alloc)
pd = atomic_rcu_read(lp);
if (pd == NULL) {
+ void *existing;
+
if (!alloc) {
return NULL;
}
pd = g_new0(PageDesc, V_L2_SIZE);
- atomic_rcu_set(lp, pd);
+ existing = atomic_cmpxchg(lp, NULL, pd);
+ if (unlikely(existing)) {
+ g_free(pd);
+ pd = existing;
+ }
}
return pd + (index & (V_L2_SIZE - 1));
diff --git a/docs/devel/multi-thread-tcg.txt b/docs/devel/multi-thread-tcg.txt
index a99b4564c6..faf8918b23 100644
--- a/docs/devel/multi-thread-tcg.txt
+++ b/docs/devel/multi-thread-tcg.txt
@@ -134,8 +134,8 @@ tb_set_jmp_target() code. Modification to the linked lists
that allow
searching for linked pages are done under the protect of the
tb_lock().
-The global page table is protected by the tb_lock() in system-mode and
-mmap_lock() in linux-user mode.
+The global page table is a lockless radix tree; cmpxchg is used
+to atomically insert new elements.
The lookup caches are updated atomically and the lookup hash uses QHT
which is designed for concurrent safe lookup.
--
2.17.1
- [Qemu-devel] [PULL v2 00/19] tcg queued patches, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 02/19] qht: require a default comparison function, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 01/19] tcg/i386: Use byte form of xgetbv instruction, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 03/19] qht: return existing entry when qht_insert fails, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 04/19] tcg: track TBs with per-region BST's, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 05/19] tcg: move tb_ctx.tb_phys_invalidate_count to tcg_ctx, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 06/19] translate-all: iterate over TBs in a page with PAGE_FOR_EACH_TB, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 08/19] translate-all: remove hole in PageDesc, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 07/19] translate-all: make l1_map lockless,
Richard Henderson <=
- [Qemu-devel] [PULL v2 09/19] translate-all: work page-by-page in tb_invalidate_phys_range_1, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 10/19] translate-all: move tb_invalidate_phys_page_range up in the file, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 11/19] translate-all: use per-page locking in !user-mode, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 13/19] translate-all: introduce assert_no_pages_locked, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 12/19] translate-all: add page_locked assertions, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 14/19] translate-all: discard TB when tb_link_page returns an existing matching TB, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 16/19] cputlb: remove tb_lock from tlb_flush functions, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 17/19] translate-all: remove tb_lock mention from cpu_restore_state_from_tb, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 15/19] translate-all: protect TB jumps with a per-destination-TB lock, Richard Henderson, 2018/06/15
- [Qemu-devel] [PULL v2 19/19] tcg: Reduce max TB opcode count, Richard Henderson, 2018/06/15