[Bug binutils/23008] Stack Overflow(Stack Exhaustion) in demangle related functions

[matz at suse dot de]

https://sourceware.org/bugzilla/show_bug.cgi?id=23008

Michael Matz <matz at suse dot de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |matz at suse dot de

--- Comment #11 from Michael Matz <matz at suse dot de> ---
All seems to work as designed.  The testcase contains a large number of 'F'
characters, and demangling each one of them entails:

5  0x00000000005ec0f8 in demangle_nested_args (work=0x7fffffffd540,
mangled=0x7fffffffd2a0, 
    declp=0x7fffff800050) at ../../libiberty/cplus-dem.c:4713
4713      result = demangle_args (work, mangled, declp);
#4  0x00000000005ea8f9 in demangle_args (work=0x7fffffffd540,
mangled=0x7fffffffd2a0, declp=0x7fffff800050)
    at ../../libiberty/cplus-dem.c:4659
4659              if (!do_arg (work, mangled, &arg))
#3  0x00000000005eb99e in do_arg (work=0x7fffffffd540, mangled=0x7fffffffd2a0,
result=0x7fffff7ffbe0)
    at ../../libiberty/cplus-dem.c:4332
4332      if (!do_type (work, mangled, work->previous_argument))
#2  0x00000000005cbf15 in do_type (work=0x7fffffffd540, mangled=0x7fffffffd2a0,
result=0x6030000318d0)
    at ../../libiberty/cplus-dem.c:3719
3719              if (!demangle_nested_args (work, mangled, &decl)
#1  0x00000000005ec0f8 in demangle_nested_args (work=0x7fffffffd540,
mangled=0x7fffffffd2a0, 
    declp=0x7fffff7ff370) at ../../libiberty/cplus-dem.c:4713
4713      result = demangle_args (work, mangled, declp);

That progresses *mangled by one character.  When compiled with clang, the above
sequence of five calls needs 3296 bytes on the stack.  The testcase
contains more than 2542 'F' characters in a row, and together that needs more
than 8MB of stack, leading to the abort.

When compiled with GCC -fsanitize-address the above sequence only needs 912
bytes on stack (per 'F' character), so it progresses until 
(gdb) p *mangled
$10 = 0x78b6cc <mbuffer+9196> 'F' <repeats 200 times>...
before segfaulting due to stack overflow (with clang it only gets until
mbuffer+2550).

When compiled without sanitizer (with GCC) the above sequence of calls only
needs 400 bytes per stack.  The testcase contains 11586 'F' characters, so that
is within the normal stack limit and no problem occurs.

If the compiler is more clever (the above is with gcc-6 and -O0) then the
sequence of calls will need less stack space, and hence not reproduce the
problem.  I'm not sure if anything needs fixing, the demangler works as
designed, you ask it to demangle a nested structure that's 11000 levels deep,
and a stack overflow occurs.  As expected.

-- 
You are receiving this mail because:
You are on the CC list for the bug.