Re: mac99 SMP

[BALATON Zoltan]

On Wed, 5 Mar 2025, Andrew Randrianasulu wrote:
> ср, 5 мар. 2025 г., 19:52 BALATON Zoltan <balaton@eik.bme.hu>:
>
> > On Wed, 5 Mar 2025, Andrew Randrianasulu wrote:
> > > > yeah, with 0x0 I can't single step. Text from documentation webpage says
> > > >
> > > > ====
> > > >
> > > > maintenance packet qqemu.sstepbits
> > > >
> > > > This will display the MASK bits used to control the single stepping IE:
> > > >
> > > > (gdb) maintenance packet qqemu.sstepbits
> > > > sending: "qqemu.sstepbits"
> > > > received: "ENABLE=1,NOIRQ=2,NOTIMER=4"
> > > >
> > > > =====
> > >
> > >
> > > Trying to resend second time
> > >
> > > I tried 0x3 and was able single step to second cpu bringup
> > >
> > > Simple 0x1 does not work (it single steps until irq_save (?) and then
> > > just go forward into hang on secondary cpu)
> >
> > So it ends in nmi_stop_this_cpu but there's no backtrace at that point to
> > see how it ended up there. Before that it was setting up L2 cache but you
> > did not step though that just hit continue so we don't know of that
> > finished or failed somewhere. Was this a run that ended with the second
> > CPU detected or not?
>
> sorry, there was whole session with few attempts.
>
> Last run (where it stopped with nmi) was success, I just run kernel alone
> and let it panic on "no root found"

So nmi_stop_this_cpu is when the CPU is started and idle. We want to know 
where is it when it says CPU1 is stuck. If CPU1 reached nmi_stop_this_cpu 
even in that case but CPU0 did not detect it then the part which signals 
CPU0 about CPU1 running should be analysed further. If CPU1 stops 
somewhere before the place where it stops should be found but all the 
traces you've sent show successful runs not when CPU1 is stuck.

> It reached this place in 5 seconds (by timestamp). In fail case it prints
> same panic message at ~30 seconds. Interestingly, it fails a bit
> differently with -accel tcg, thread=single and -smp 2. Much longer delay
> and it reports soft lockup in guest kernel and not fully booted up. So for
> now all those runs with thread=multi !

I guess it's normal it takes longer to emulate two CPUs with one host CPU 
than letting them run on two host CPUs. The timing and synchronisation 
might be different in the two cases. If the error is different maybe that 
gives a clue where it stops but could also be some bug with single 
threaded TCG which I think was not tested too much lately.

> For some reason even if in dbg I see printk from l2cr printing line it does
> not reach qemu  console in another terminal at the same time / step , so I
> guess even printing to console from kernel threads not fully synchronous ...

If it always reaches L2 setup it could be it's waiting for some bit in one 
of the cache control registers. There are comments in that code in Linux 
saying that and I think QEMU does not emulate those registers so you could 
try adding some debug logs or enable traces in QEMU on accessing those 
regs and see if the bits are correctly emulated. But I can't explain why 
that would only be a problem when not stepping through it and how could 
gdb avoid it so maybe it's something else. I don't know what's the problem 
but have no better idea from the info we have so far. If you can find more 
exactly where it's stuck in the case when it's not working then we could 
have a better understanding on what to check further. Or you could add a 
breakpoint after L2 setup is done and see if that's reached then move the 
breakpoint later and see if that's reached and so on. It's not even clear 
if in the case when it reports CPU1 is stuck is CPU1 really stuck or only 
CPU0 did not notice it ran because maybe there's some problem with the way 
it signals CPU0 that does not always work correctly under QEMU. So you 
should check if CPU1 reaches the place it should reach or stops somewhere 
before that then if it did run check why CPU0 could miss to find the flag 
it should set.

> I am not sure how to attack this? Set watchpoint? But on what exactly? It
> also probably will be slow(gdb itself run under emulation)

gdb itself is not run under emulation, it's observing the emulated system 
from outside. It would be run in emulation if you started gdb under Linux 
in the guest.

Regards,
BALATON Zoltan