Re: [Qemu-devel] ARM brk bug

[Alexander Graf]

On 02.03.2012, at 18:49, Peter Maydell wrote:

> On 27 February 2012 15:16, Bernhard M. Wiedemann <address@hidden> wrote:
>> I found that running a debian arm5 bash with qemu runs into varying
>> problems with -R but works without.
> 
> So I had a look at this this afternoon, and what seems to be happening
> is that with -R, the call to target_mmap() in elfload.c:setup_arg_pages()
> (which creates the stack) is putting the stack immediately after the
> bash BSS segment in the address space. This means that brk() will
> never be able to expand, and it looks like something in either bash
> or libc's locale code isn't correctly handling the failure, so we
> crash. (The segfault is from a strlen(NULL) from setlocale() I think.)
> 
> We should probably try to put the stack somewhere more sensible than
> where it currently ends up...

I wrote a small test case to reproduce the breakage as far as I understood it 
at least:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/fcntl.h>

int main(int argc, char **argv)
{
    void *curbrk;
    void *tmp;
    char buf[1024];
    int fd, r;

    tmp = mmap((void*)0x5000UL, 0x10000, PROT_READ|PROT_WRITE,
               MAP_PRIVATE|MAP_ANONYMOUS, 0, 0);
    printf("mmap: %p\n", tmp);

    curbrk = sbrk(8 * 1024 * 1024);
    printf("current brk: %p\n", curbrk);

    fd = open("/proc/self/maps", O_RDONLY);
    while ((r = read(fd, buf, sizeof(buf))) > 0) {
        write(1, buf, r);
    }

    return 0;
}

On real hw:

$ ./brk
mmap: 0x40056000
current brk: 0x12000
00008000-00009000 r-xp 00000000 b3:02 29374      /root/brk-bug/brk
00010000-00011000 r--p 00000000 b3:02 29374      /root/brk-bug/brk
00011000-00012000 rw-p 00001000 b3:02 29374      /root/brk-bug/brk
00012000-00812000 rw-p 00000000 00:00 0          [heap]
40014000-40015000 rw-p 00000000 00:00 0 
40035000-4004d000 r-xp 00000000 b3:02 6823       /lib/ld-2.15.so
4004d000-4004e000 rw-p 00000000 00:00 0 
40054000-40055000 r--p 00017000 b3:02 6823       /lib/ld-2.15.so
40055000-40056000 rw-p 00018000 b3:02 6823       /lib/ld-2.15.so
40056000-40066000 rw-p 00000000 00:00 0 
4008e000-4008f000 rw-p 00000000 00:00 0 
400a8000-400b1000 r-xp 00000000 b3:02 6846       /lib/libgcc_s.so.1
400b1000-400b8000 ---p 00009000 b3:02 6846       /lib/libgcc_s.so.1
400b8000-400b9000 r--p 00008000 b3:02 6846       /lib/libgcc_s.so.1
400b9000-400ba000 rw-p 00009000 b3:02 6846       /lib/libgcc_s.so.1
40150000-4022d000 r-xp 00000000 b3:02 6830       /lib/libc-2.15.so
4022d000-40234000 ---p 000dd000 b3:02 6830       /lib/libc-2.15.so
40234000-40236000 r--p 000dc000 b3:02 6830       /lib/libc-2.15.so
40236000-40237000 rw-p 000de000 b3:02 6830       /lib/libc-2.15.so
40237000-4023a000 rw-p 00000000 00:00 0 
bee3f000-bee60000 rw-p 00000000 00:00 0          [stack]
ffff0000-ffff1000 r-xp 00000000 00:00 0          [vectors]

strace says:
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 
0x40082000
[...]
mmap2(0x5000, 65536, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, 0, 0) = 
0x40018000
[...]
brk(0)                                  = 0x12000
brk(0x812000)                           = 0x812000

In QEMU with -R <high>

$ qemu-arm -R $(( 0x10000000 )) ./brk
mmap: 0x935000
current brk: 0xffffffff
00000000-00008000 ---p 00000000 00:00 0
00008000-00009000 r-xp 00000000 08:09 1248935408          /brk
00009000-00010000 ---p 00000000 00:00 0
00010000-00011000 r--p 00000000 08:09 1248935408          /brk
00011000-00012000 rw-p 00001000 08:09 1248935408          /brk
00012000-00013000 ---p 00000000 00:00 0
00013000-00813000 rw-p 00000000 00:00 0
00813000-0082b000 r-xp 00000000 08:09 1248675559          /lib/ld-2.15.so
0082b000-00832000 ---p 00000000 00:00 0
00832000-00833000 r--p 00017000 08:09 1248675559          /lib/ld-2.15.so
00833000-00834000 rw-p 00018000 08:09 1248675559          /lib/ld-2.15.so
00834000-00835000 rw-p 00000000 00:00 0
00835000-00838000 ---p 00000000 00:00 0
00838000-00915000 r-xp 00000000 08:09 1248675566          /lib/libc-2.15.so
00915000-0091c000 ---p 000dd000 08:09 1248675566          /lib/libc-2.15.so
0091c000-0091e000 r--p 000dc000 08:09 1248675566          /lib/libc-2.15.so
0091e000-0091f000 rw-p 000de000 08:09 1248675566          /lib/libc-2.15.so
0091f000-00922000 rw-p 00000000 00:00 0
00922000-0092b000 r-xp 00000000 08:09 1248678464          /lib/libgcc_s.so.1
0092b000-00932000 ---p 00009000 08:09 1248678464          /lib/libgcc_s.so.1
00932000-00933000 r--p 00008000 08:09 1248678464          /lib/libgcc_s.so.1
00933000-00934000 rw-p 00009000 08:09 1248678464          /lib/libgcc_s.so.1
00934000-00946000 rw-p 00000000 00:00 0
00013000-00813000 rw-p 00000000 00:00 0          [stack]

qemu's strace says:
15545 mmap2(NULL,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 
0x00834000
[...]
15545 
mmap2(0x00005000,65536,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,0,0) = 
0x00935000
[...]
15545 brk(NULL) = 0x00012000
15545 brk(0x00812000) = 0x00012000

I wonder who allocates the chunk from 00012000-00013000? I don't see any 
syscall returning an address in that range on the trace.

Full trace: http://paste.debian.net/158450/

In QEMU with -R 0

$ qemu-arm -R 0 ./brk
mmap: 0xffffffff
current brk: 0x12000
00008000-00009000 r-xp 00000000 08:09 1248935408          /brk
00009000-00010000 ---p 00000000 00:00 0
00010000-00011000 r--p 00000000 08:09 1248935408          /brk
00011000-00012000 rw-p 00001000 08:09 1248935408          /brk
00012000-00812000 rw-p 00000000 00:00 0
0fff8000-101b8000 r-xp 00000000 08:09 1248674338          /usr/bin/qemu-arm
102b8000-102e7000 rw-p 002c0000 08:09 1248674338          /usr/bin/qemu-arm
102e7000-122e8000 rwxp 00000000 00:00 0
122e8000-12336000 rw-p 00000000 00:00 0
12336000-12337000 rwxp 00000000 00:00 0
12337000-12370000 rw-p 00000000 00:00 0          [heap]
18000000-18001000 ---p 00000000 00:00 0
18001000-18801000 rw-p 00000000 00:00 0
18801000-18819000 r-xp 00000000 08:09 1248675559          /lib/ld-2.15.so
18819000-18820000 ---p 00000000 00:00 0
18820000-18821000 r--p 00017000 08:09 1248675559          /lib/ld-2.15.so
18821000-18822000 rw-p 00018000 08:09 1248675559          /lib/ld-2.15.so
18822000-18823000 rw-p 00000000 00:00 0
18826000-18903000 r-xp 00000000 08:09 1248675566          /lib/libc-2.15.so
18903000-1890a000 ---p 000dd000 08:09 1248675566          /lib/libc-2.15.so
1890a000-1890c000 r--p 000dc000 08:09 1248675566          /lib/libc-2.15.so
1890c000-1890d000 rw-p 000de000 08:09 1248675566          /lib/libc-2.15.so
1890d000-18910000 rw-p 00000000 00:00 0
18910000-18919000 r-xp 00000000 08:09 1248678464          /lib/libgcc_s.so.1
18919000-18920000 ---p 00009000 08:09 1248678464          /lib/libgcc_s.so.1
18920000-18921000 r--p 00008000 08:09 1248678464          /lib/libgcc_s.so.1
18921000-18922000 rw-p 00009000 08:09 1248678464          /lib/libgcc_s.so.1
18922000-18924000 rw-p 00000000 00:00 0
ffff0000-ffff1000 r--p 00000000 00:00 0
18001000-18801000 rw-p 00000000 00:00 0          [stack]

qemu's strace says:
15659 mmap2(NULL,4096,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,-1,0) = 
0x18822000
[...]
15659 
mmap2(0x00005000,65536,PROT_READ|PROT_WRITE,MAP_PRIVATE|MAP_ANONYMOUS,0,0) = 
0xfffffff4
[...]
15659 brk(NULL) = 0x00012000
15659 brk(0x00812000) = 0x00812000

Full trace: http://paste.debian.net/158451/

Don't be surprised about the duplicate [stack] entry at the end. I manually add 
that to the /proc/self/stat output at the end of the real sanitized 
/proc/self/maps data from the host in my maps hijack code.


Alex