Re: [Libunwind-devel] Bug fixes in libunwind git = make another "release"?

[Lassi Tuura]

Hi,

>> One of the things that continues to affect us is the introduction of 
>> mincore() call vs. msync() for address validation: mincore() gives an 
>> incorrect answer for MAP_PRIVATE mappings for linux kernels prior to 2.6.21. 
>> Can we make libunwind auto-detect whether mincore() works and fall back on 
>> msync(), or at least integrate the patch to make it configure-time option?
> 
> I'll add a configure time check for this unless someone else beats me to it.

See http://article.gmane.org/gmane.comp.lib.unwind.devel/728; I had some ideas 
how to test it at run-time, will post code if my tests work.

>> Another issue continuing to bite us from time to time is the deadlock in 
>> dl_iterate_phdr() if you attempt to get a stack trace right in the middle of 
>> dynamic linker holding - or even taking - that lock. Previously it was 
>> suggested app might trap into _dl_debug_state() to grab the list then tell 
>> libunwind to use the list not dl_iterate_phdr(). I am willing to see if this 
>> can fly, but thoughts would be welcome.
> 
> I wish we could convince glibc developers to implement what we need (a
> lock free implementation of a dl_iterate_phdr() like interface).

Well, perhaps we can try asking once more. I can give it a go when I'm done 
with my other pending work.

>> We build our libunwind with --disable-block-signals, which turns off 
>> sigprocmask() calls with mutexes as they cost a bit and aren't needed in our 
>> case. How would we recommend a distro build libunwind?
> 
> My guess is that a higher level API such as src/mi/backtrace.c is
> sufficient for most users. As long as we ensure signal safety in such
> a higher level API, we could make --disable-block-signals the default
> (with some man page, documentation updates of course).

I don't know enough about what use case the sigprocmask() calls serve, so can't 
comment much. At the moment the fast trace yields a trace in <3000 clock cycles 
(~100 cycles per stack level). This is already at pain threshold for us, so 
would prefer it not slowing down :-)

>> Should we advise distros that libunwind is dangerous in security sensitive 
>> code / user apps unless the entire userland has been compiled with GCC 4.5+? 
>> The invalid unwind info is a large attack vector... Earlier we enabled full 
>> address validation at least on x86-64 - but the fast trace code runs with 
>> validation off. Even with validation I am somewhat concerned this gets used 
>> to attack apps, bring down servers, or such.
> 
> The security situation should improve If we stop unwinding when there
> is missing dwarf information and identify signal frames only by
> augmentation attribute (essentially turning off frame chain based
> unwinding). Do you have any data on what percentage of the frames are
> lost if we do this on a gcc-4.5+ setup?

With GCC <4.5 it's the functions _with_ unwind info that are most dangerous :-) 
But yeah, turning off the heuristic code would help.

As for data, I've never run our software on a system with everything (libc, 
libm, nss switch etc.) compiled with GCC 4.5+. With all but basic system 
libraries compiled with GCC 4.5+, for a program which runs for ~20 minutes of 
CPU time, we seem to get 1.0% stack walks terminated early. 0.4% is because 
_init doesn't have a walkable frame chain: GCC crt* files need recompiling with 
unwind info.

In total the profile I looked at has 679 functions reported under 
"<spontaneous>", out of 14072 functions reported. Majority of the addresses are 
just "junk", with no symbol association, some even invalid code addresses like 
0x1. I'd guess half to three quarters of this junk is from bad unwind info in 
libm. I've not re-analysed lately the causes for the rest - we do use an 
interpreter which calls to native code, and it's possible it does things 
libunwind doesn't understand.

I have a vague recollection there is at least one more place in addition to 
crt* where GCC is still omitting the frame info. Some special case virtual 
thunks, alternate/cloned constructors, or something.

IIRC on RHEL5/x86_64 the frame chain also ends at _start, __libc_start_main end 
of things with one or two frames of missing unwind info. Something there needs 
special heuristic handling - initially the fast trace gave up on frames without 
unwind info, but I had to make it understand the successful heuristic walks.

Regards,
Lassi