[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v2] target/ppc: cpu_init: Clean up stop state on cpu reset

From: Cédric Le Goater
Subject: Re: [PATCH v2] target/ppc: cpu_init: Clean up stop state on cpu reset
Date: Sat, 18 Jun 2022 13:35:38 +0200
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.9.0

On 6/17/22 11:52, Frederic Barrat wrote:
The 'resume_as_sreset' attribute of a cpu is set when a thread is
entering a stop state on ppc books. It causes the thread to be
re-routed to vector 0x100 when woken up by an exception. So it must be
cleared on reset or a thread might be re-routed unexpectedly after a
reset, when it was not in a stop state and/or when the appropriate
exception handler isn't set up yet.

Using skiboot, it can be tested by resetting the system when it is
quiet and most threads are idle and in stop state.

After the reset occurs, skiboot elects a primary thread and all the
others wait in secondary_wait. The primary thread does all the system
initialization from main_cpu_entry() and at some point, the
decrementer interrupt starts ticking. The exception vector for the
decrementer interrupt is in place, so that shouldn't be a
problem. However, if that primary thread was in stop state prior to
the reset, and because the resume_as_sreset parameters is still set,
it is re-routed to exception vector 0x100. Which, at that time, is
still defined as the entry point for BML. So that primary thread
restarts as new and ends up being treated like any other secondary
thread. All threads are now waiting in secondary_wait.

It results in a full system hang with no message on the console, as
the uart hasn't been init'ed yet. It's actually not obvious to realise
what's happening if not tracing reset (-d cpu_reset). The fix is
simply to clear the 'resume_as_sreset' attribute on reset.

Signed-off-by: Frederic Barrat <fbarrat@linux.ibm.com>
v2: rework commit message

Nice ! This has been a long standing bug. I chased it for weeks.
I was reproducing with intensive I/Os, doing an scp on an emulated
PowerNV machine. It hung after a while (unless using powersave=off)

Now, with this patch, a QEMU PowerNV POWER9 machine (SMP) running a
Linux 5.18 sustains the load :

  $ scp ./ubuntu-22.04-ppc64le.qcow2 root@vm103:/dev/null
  root@vm103's password:
  ubuntu-22.04-ppc64le.qcow2                    100% 8581MB   5.8MB/s   24:39

Quite a few interrupts :

  # grep PNV-PCI-MSI  /proc/interrupts
   51:          9          0  PNV-PCI-MSI 403177472 Edge      nvme0q0
   52:          2          0  PNV-PCI-MSI 403177473 Edge      nvme0q1
   53:          0          0  PNV-PCI-MSI 403177474 Edge      nvme0q2
   54:    3427556          0  PNV-PCI-MSI 135315456 Edge      eth0-rx-0
   55:          0    4261742  PNV-PCI-MSI 135315457 Edge      eth0-tx-0
   56:          1          0  PNV-PCI-MSI 135315458 Edge      eth0
   57:          0         71  PNV-PCI-MSI 135299072 Edge      xhci_hcd
   58:          0          0  PNV-PCI-MSI 135299073 Edge      xhci_hcd
   59:          0          0  PNV-PCI-MSI 135299074 Edge      xhci_hcd

It would be nice to explain what you did to corner the issue. It would
help other people chasing similar bugs in QEMU or in the kernel.



  target/ppc/cpu_init.c | 3 +++
  1 file changed, 3 insertions(+)

diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 0f891afa04..c16cb8dbe7 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7186,6 +7186,9 @@ static void ppc_cpu_reset(DeviceState *dev)
+    /* clean any pending stop state */
+    env->resume_as_sreset = 0;
      env->reserve_addr = (target_ulong)-1ULL;

reply via email to

[Prev in Thread] Current Thread [Next in Thread]