Re: ***UNCHECKED*** Re: mac99 SMP

[Andrew Randrianasulu]

On Tue, Mar 4, 2025 at 6:56 PM BALATON Zoltan <balaton@eik.bme.hu> wrote:
>
> On Tue, 4 Mar 2025, Andrew Randrianasulu wrote:
> > On Tue, Mar 4, 2025 at 6:09 PM Andrew Randrianasulu
> > <randrianasulu@gmail.com> wrote:
> >>
> >> On Tue, Mar 4, 2025 at 4:17 PM BALATON Zoltan <balaton@eik.bme.hu> wrote:
> >>>
> >>> On Tue, 4 Mar 2025, Andrew Randrianasulu wrote:
> >>>> I think this is maximum, I added few more printks, you can see them in
> >>>> earlier log. But sadly it detours into assembly and all this livepatching
> >>>> ....
> >>>
> >>> See linux/Documentation/admin-guide/kernel-parameters.txt loglevel=7 or
> >>> ignore_loglevel should be the maxumum level but it seems it's in the
> >>> assembly part where there are no more detailed logs printed.
> >>>
> >>>> Can qemu only print traces if instructed so from monitor or even remote 
> >>>> gdb
> >>>> ? Otherwise logs will be huge ...
> >>>
> >>> You can enable traces from QEMU monitor but not debug options I think.
> >>>
> >>>> I trued to just add -d mmu and log was overwhelming ...
> >>>
> >>> Yes this is a lot of logs, redirect to a file with &> then cut out the
> >>> interesting part. Especially when it's stuck you would see a lot of ISIs
> >>> repeating so you can stop at that point, no need to wait until it finds
> >>> the CPU stuck, we just need the first few of those ISI logs. What I'm
> >>> wondering if when stepping through are these don't happen or the MMU gets
> >>> set up before that happens? Or maybe it has something to do with the DECR
> >>> interrupt CPU0 gets but gdb handles somehow, it prints a message about
> >>> that and the ISIs on CPU1 seem to start at that point so maybe that
> >>> changes something somewhere that causes CPU1 to start getting interrupts
> >>> too when it should not yet?
> >>
> >>
> >> Well, logs are at ~100MB each, I tried to diff two of them (both cases
> >> were run  with -S -s params, so remote gdb was always there, just in
> >> one case it was just executing without breakpoint set)
> >>
> >>  ./qemu-system-ppc -d int,mmu -M mac99,via=pmu -cpu g4    -cdrom
> >> ~/ISO/install-powerpc-universal-2008.0.iso    -boot d -g 1024x768x8
> >> -smp 2  -bios ~/K38_sdcard1/Documents/openbios-qemu-smp.elf -display
> >> sdl -accel tcg,thread=multi -m 2G -kernel ~/boot/vmlinux-6.12.17
> >> -append "console=ttyPZ0" -S -s  > LOG4.mmu  2>&1
> >>
> >> ./qemu-system-ppc -d int,mmu -M mac99,via=pmu -cpu g4    -cdrom
> >> ~/ISO/install-powerpc-universal-2008.0.iso    -boot d -g 1024x768x8
> >> -smp 2  -bios ~/K38_sdcard1/Documents/openbios-qemu-smp.elf -display
> >> sdl -accel tcg,thread=multi -m 2G -kernel ~/boot/vmlinux-6.12.17
> >> -append "console=ttyPZ0" -S -s  > LOG3.mmu  2>&1
> >>
> >> LOG3 was with breakpoint LOG4 without
> >>
> >> Diff between them is like 9.5 MB, I see no ISI there, but DECR appear ...
> >>
> >> Sorry, I just do not know how to interpret this. :(
> >
> > They compres extremely well, down to less than ! mb, so attaching them here 
> > ...
>
> Better not send large attachments to lists with lot of subscribers. Upload
> it somewhere and post the URL only.

sorry, will do that next time.


These are just the QEMU debug logs but
> missing the linux logs inbetween so we can't see where is the relevant
> part. Keep -serial stdio and also -append "console=ttyPZ0 debug
> ignore_loglevel" then you can see what part of the log to cut which is the
> interesting part around and after smp_kick. You don't seem to get the same
> issue I saw with the Ubuntu image JD used so maybe this newer Linux
> version has some other issue. I have no idea what and if it can't be
> reproduced when stepping through with gdb then I don't know how to find
> the place where it stops. You could still try getting backtrace of the
> vCPUs when it's waiting for the second CPU when you have no breakpoint set
> to see what code is ececuted at that point, then look at the sources in
> Linux to see what it tries to do (if it's in assembly you'll have to learn
> enough of that to undrestand it).

I tried to set breakpoint to assembly label __start_scondary_pmac_0
and while breakpoint can be set it never hit during normal execution
(no additional breakpoints)

I also run into problems with getting both stdio and -d output into same file

 ./qemu-system-ppc -M mac99,via=pmu -cpu g4   -smp 2  -bios
~/K38_sdcard1/Documents/openbios-qemu-smp.elf -accel tcg,thread=multi
-m 2G -kernel ~/boot/vmlinux-6.12.17 -append "console=ttyPZ0" -vga
none -serial stdio  -d int,mmu 2>&1 > LOG6.mm

this line leaves only

s>> et_property: NULL phandle

>> =============================================================
>> OpenBIOS 1.1 [Mar 1 2025 05:50]
>> Configuration device id QEMU version 1 machine id 1
>> CPUs: 2
>> Memory: 2048M
>> UUID: 00000000-0000-0000-0000-000000000000
>> CPU type PowerPC,G4
>> CPU type PowerPC,G4
milliseconds isn't unique.
Output device screen not found.
Output device screen not found.
>> [ppc] Kernel already loaded (0x01000000 + 0x01760a6c) (initrd 0x00000000 + 
>> 0x00000000)
>> [ppc] Kernel command line: console=ttyPZ0
>> switching to new context:

in log :(

trying with -nographic and keeping -serial stdio qemu complain

 ./qemu-system-ppc -M mac99,via=pmu -cpu g4    -cdrom
~/ISO/install-powerpc-universal-2008.0.iso    -boot d -g 1024x768x8
-smp 2  -bios ~/K38_sdcard1/Documents/openbios-qemu-smp.elf -display
sdl -accel tcg,thread=multi -m 2G -kernel ~/boot/vmlinux-6.12.17
-append "console=ttyPZ0" -nographic -serial stdio -d int,mmu 2>&1 >
LOG6.mm
qemu-system-ppc: -accel tcg,thread=multi: warning: Guest not yet
converted to MTTCG - you may get unexpected results
qemu-system-ppc: -serial stdio: cannot use stdio by multiple character devices
qemu-system-ppc: -serial stdio: could not connect serial device to
character backend 'stdio'

so I am not sure how to mix both console output (from kernel) and
debug traces from -d switch.

I did another gdb.txt (without quitting gdb it appends to log file)

it shows cpu1 coming out of halt, but I have no idea where exactly it
start to behave differently from normal run

may be reset vector is not set?

https://elixir.bootlin.com/linux/v6.12.17/source/arch/powerpc/kernel/head_book3s_32.S#L799

there it says

/* Copy some CPU settings from CPU 0 */

I thought may be some instructions need qemu barrier of sort, there
was fix in 2024

https://patchwork.ozlabs.org/project/qemu-devel/cover/20240405125340.380828-1-npiggin@gmail.com/

found via searching for async_safe_run_on_cpu mentioned in

https://www.qemu.org/docs/master/devel/multi-thread-tcg.html

but unfortunately this is way above my league ...



>
> Regards,
> BALATON Zoltan

gdb-more2.txt
Description: Text document