[Qemu-devel] linux-user crashes on clone(2) when run on ppc host

[Emilio G. Cota]

Hi,

I'm having trouble running a simple multithreaded program on a PowerPC host 
machine.

The machine I'm using is a ppc VM--I think it's running under KVM (I'm using
OVH's RunAbove Power8 service):
  address@hidden:~/qemu$ uname -a
  Linux adsf 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:27:09 UTC 2014 
ppc64le ppc64le ppc64le GNU/Linux

The original program I tried was doing pthread_create, and it was segfaulting.
Then I distilled it to a simpler test program taken from
  https://lists.gnu.org/archive/html/qemu-devel/2005-10/msg00251.html
, which is simply doing a clone(2):

/* gcc -O0 -g -o foo foo.c -pthread -static */
#define _GNU_SOURCE
#include <sched.h>
#include <signal.h>
#include <stdio.h>
#include <sys/types.h>

int thread_main(void *arg)
{
    printf("child: Hello world!\n");
    while(1)
        ;
    return 0;
}

unsigned long stack[8192];

int main()
{
    int pid;

    printf("About to clone: thread_main=%p\n", thread_main);
    pid = clone(thread_main, stack+4096, CLONE_VM, NULL);
    if(pid == -1) {
        perror("clone");
        return 1;
    }
    printf("parent: clone successful; child pid is %d\n", pid);
    printf("parent: sleeping a bit\n");
    sleep(2);
    printf("parent: killing process\n");
    kill(pid, SIGTERM);
    return 0;
}

Doesn't work (linux-user on ppc64le host):
- x86_64 static binary, compiled natively
- ppc static binary, cross-compiled from x86 host
- ppc64le static binary, compiled natively on the ppc64le host
- ppc64le binary (i.e. non-static), compiled natively on the ppc64le host
- ppc64 binary, compiled natively on ppc64 host (running ppc64-linux-user)

Works:
- Any of the above running on x86_64 host (linux-user or native)
- ppc64le binary running natively on ppc64le host

The current HEAD of the tree is:
  commit 93f6d1c16036aaf34055d16f54ea770fb8d6d280
  Merge: 4316536 7a4dfd1
  Author: Peter Maydell <address@hidden>
  Date:   Tue Jun 16 10:35:43 2015 +0100

I've tried older versions of qemu (e.g. v2.0, v1.7) and they don't work either.

The segfault for the ppc64le static binary is as follows:

address@hidden:~/qemu$ ppc64le-linux-user/qemu-ppc64le foo
About to clone: thread_main=0x100008f0
Invalid data memory access: 0x00003fffa2f8a720
NIP 00000040009aeec8   LR 0000000010000660 CTR 00000040009aee68 XER 
0000000000000000 CPU#1
MSR 8000000002806001 HID0 0000000000000000  HF 0000000002806001 idx 0
TB 00000000 00000000
GPR00 0000000000000078 0000000010019030 0000004000a52800 0000000000000000
GPR04 0000000010019030 0000000000000027 0000000000000000 0000000000000001
GPR08 0000000000000000 0000000000000001 0000000000000000 0000000000000007
GPR12 00000040009aee68 0000004000a57b60 0000000000000000 0000000000000000
GPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR24 0000000000000000 0000000000000000 0000000000000000 000000400084be10
GPR28 000000400084c148 0000000000000100 00000000100008f0 0000000000000000
CR 42000884  [ G  E  -  -  -  L  L  G  ]             RES ffffffffffffffff
FPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000
Invalid segfault errno (42000000)
NIP 00000040009aeec8   LR 0000000010000660 CTR 00000040009aee68 XER 
0000000000000000 CPU#1
MSR 8000000002806001 HID0 0000000000000000  HF 0000000002806001 idx 0
TB 00000000 00000000
GPR00 0000000000000078 0000000010019030 0000004000a52800 0000000000000000
GPR04 0000000010019030 0000000000000027 0000000000000000 0000000000000001
GPR08 0000000000000000 0000000000000001 0000000000000000 0000000000000007
GPR12 00000040009aee68 0000004000a57b60 0000000000000000 0000000000000000
GPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
GPR24 0000000000000000 0000000000000000 0000000000000000 000000400084be10
GPR28 000000400084c148 0000000000000100 00000000100008f0 0000000000000000
CR 42000884  [ G  E  -  -  -  L  L  G  ]             RES ffffffffffffffff
FPR00 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR04 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR08 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR12 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR16 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR20 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR24 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPR28 0000000000000000 0000000000000000 0000000000000000 0000000000000000
FPSCR 0000000000000000

^C

gdb stack trace:
[...]
Using host libthread_db library "/lib/powerpc64le-linux-gnu/libthread_db.so.1".
[New Thread 0x3fffb7aaf170 (LWP 12287)]
About to clone: thread_main=0x100007f4
[New Thread 0x3fffb3a7f170 (LWP 12288)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x3fffb3a7f170 (LWP 12288)]
0x0000000060014828 in ppc_tb_set_jmp_target (jmp_addr=<error reading variable: 
Cannot access memory at address 0x3fffb3a2a748>, addr=<error reading variable: 
Cannot access memory at address 0x3fffb3a2a740>) at 
/home/admin/qemu/tcg/ppc/tcg-target.c:1247
1247    {
(gdb) bt
#0  0x0000000060014828 in ppc_tb_set_jmp_target (jmp_addr=<error reading 
variable: Cannot access memory at address 0x3fffb3a2a748>, addr=<error reading 
variable: Cannot access memory at address 0x3fffb3a2a740>) at 
/home/admin/qemu/tcg/ppc/tcg-target.c:1247
#1  0x0000000060009ce0 in tb_set_jmp_target (tb=0x3fffb3adf4f0, n=0, 
addr=1614371232) at /home/admin/qemu/include/exec/exec-all.h:286
#2  0x000000006000b648 in tb_reset_jump (tb=0x3fffb3adf4f0, n=0) at 
/home/admin/qemu/translate-all.c:907
#3  0x000000006000c5d0 in tb_link_page (tb=0x3fffb3adf4f0, 
phys_pc=274888060616, phys_page2=18446744073709551615) at 
/home/admin/qemu/translate-all.c:1363
#4  0x000000006000bccc in tb_gen_code (cpu=0x6241b570, pc=274888060616, 
cs_base=0, flags=41967617, cflags=0) at /home/admin/qemu/translate-all.c:1034
#5  0x000000006000e6f0 in tb_find_slow (env=0x62423990, pc=274888060616, 
cs_base=0, flags=41967617) at /home/admin/qemu/cpu-exec.c:299
#6  0x000000006000ea14 in tb_find_fast (env=0x62423990) at 
/home/admin/qemu/cpu-exec.c:327
#7  0x000000006000efe4 in cpu_ppc_exec (env=0x62423990) at 
/home/admin/qemu/cpu-exec.c:485
#8  0x00000000600716e4 in cpu_loop (env=0x62423990) at 
/home/admin/qemu/linux-user/main.c:1569
#9  0x0000000060083c50 in clone_func (arg=0x3fffffffcbb8) at 
/home/admin/qemu/linux-user/syscall.c:4536
#10 0x00003fffb7cc89d8 in start_thread (arg=0x3fffb3a7f170) at 
pthread_create.c:314
#11 0x00003fffb7c1ef00 in clone () at 
../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:104


Can you reproduce this on a real host? I wonder whether the fact that the
host here is a VM has to do with it.

Thanks,

                Emilio