[Gnewsense-dev] Re: Debugging linux freeze / crash problems on the yeeloong

[Daniel Clark]

Wu Zhangjin wrote:
> Hi,
> 
> On Tue, 2010-01-05 at 16:00 -0500, Danny Clark wrote:
>> Could someone speak to how to debug problems on the yeeloong, where the
>> kernel, linux freezes without spewing any errors to the console?
> 
> are you working with the console? and with the 2D acceleration of sm712
> enabled? If yes, the problem should be the bug of 2D acceleration.

I've reproduced from other boxes via ssh into the target box as well, so
I do not think this is the problem I was seeing; but I think we
currently do have CONFIG_FB_SM7XX_ACCEL enabled, so it is important that
we disable it judging from your information.

> We have found a bug in the 2D acceleration of sm7xx video driver, it may
> make the whole system hang when scrolling the screen, for example, if we
> copy some files from another machines to yeeloong via scp and the screen
> output is enabled, the whole system may hang.
> 
> Before this bug is really fixed, please ensure CONFIG_FB_SM7XX_ACCEL is
> disabled:
> 
> $ gunzip < /proc/config.gz | grep SM7XX
> CONFIG_FB_SM7XX=y
> # CONFIG_FB_SM7XX_ACCEL is not set
> 
>> I found Wu's paper at
>> http://lwn.net/images/conf/rtlws11/papers/proc/p14.pdf and we are trying
>> ftrace,
> 
> If you want to play with ftrace, there are two good documents from the
> original author of ftrace:
> 
> Debugging the kernel using Ftrace - part 1
> http://lwn.net/Articles/365835/
> Debugging the kernel using Ftrace - part 2 (subscriber-only)
> http://lwn.net/Articles/366796/
> 
>>  but it would be useful to know exactly what tools Lemote used
>> early in the bring-up process - for example people have mentioned that
>> it is possible to do linux debugging using PMON, but I've been unable to
>> find any documentation on how to do that.
>>
> 
> You can get the documentation from
> 
> PMON> h
> 
> find out the Debugger and Memory part, which will give you some help.
> 
>> One specific thing we would like to know is if anyone has gotten kgdb to
>> work on the system - Bernie thought that maybe at the momment mips big
>> endien was supported, but mips little endian (mipsel), as the loongson2f
>> is, was not.
> 
> kgdb really works, besides, serial port debugging is a good choice when
> the system is in the progress of booting(but for YeeLoong, it should not
> be a good choice, except we "break" the machine and touch the serial
> port pins ourselves):
> 
> # early printk will help a lot before the console is initialized
> CONFIG_EARLY_PRINTK=y
> 
> # serial port driver is useful with console=ttyS0,115200 ... 
> CONFIG_SERIAL_8250=y
> 
> and also, netconsole is another choice(after the network is
> initialized):
> 
> Device Drivers -->
>             
> [*] Network device support  -->
> [*] Network console logging support
> 
> and the watch register(the 18th register of cp0) of loongson can be used
> to trace the read/write to a virtual address:
> 
>                             63  3   2  1  0
>                             VADDR,  0, R, W
> 
> Set(whath an address）: load t0, (VADDR | 0RW); mtc0 t0, 18
> Clear: mtc0 $ZERO, 18
> 
> and there are also some other methods, including the kernel APIs: BUG,
> WARN_ON, show_registers, show_regs, dump_stack/show_trace/show_stack,
> print_symbol, these APIs may help to debug the kernel oops.
> 
> and the "magic sysrq" may help to debug the system freeze, the oprofile,
> ftrace and kgcov may help to improve the performance of the system.
> 
> Regards,
>       Wu Zhangjin


-- 
Daniel JB Clark   | Sys Admin, Free Software Foundation
pobox.com/~dclark | http://www.fsf.org/about/staff#danny