Re: qemu-ppc64-linux-user: signal handler endianness in elfv2

[Laurent Vivier]

Le 08/02/2020 à 19:57, Vincent Fazio a écrit :
> I ran into a crash scenario in ppc64-linux-user when trying to run
> openssl compiled with gcc & musl which emits an ELFv2 binary. The target
> CPU was a big endian model (e6500)
> 
> Example:
> 
>>>>>>
> vfazio@vfazio1 ~/development/buildroot/output/target :( $ gdb
> ~/development/qemu/ppc64-linux-user/qemu-ppc64
> (gdb) run -d guest_errors -cpu e6500 -E
> LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> lib/libc.so bin/bash
> Starting program:
> /home/vfazio/development/qemu/ppc64-linux-user/qemu-ppc64 -d
> guest_errors -cpu e6500 -E
> LD_LIBRARY_PATH="/home/vfazio/development/buildroot/output/target/lib/:/home/vfazio/development/buildroot/output/target/usr/lib/"
> lib/libc.so bin/bash
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
> [New Thread 0x7ffff7ff9700 (LWP 27186)]
> invalid/unsupported opcode: 1f - 0c - 1a - 00 (7c005698) 0000000010028e58 0
> 
> Thread 1 "qemu-ppc64" received signal SIGSEGV, Segmentation fault.
> 0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> /home/vfazio/development/qemu/include/qemu/bswap.h:351
> 351        __builtin_memcpy(&r, ptr, sizeof(r));
> 
> (gdb) !readelf -h bin/bash | grep abi
>   Flags:                             0x2, abiv2
> (gdb) !readelf -h lib/libc.so | grep abi
>   Flags:                             0x2, abiv2
> <<<<<
> 
> openssl does "probing" for PPC features (Altivec, Crypto, etc) by trying
> to execute possibly illegal instructions and handling them... hence that
> SIGILL. The problem is when it comes to jumping to the signal handler
> 
>>>>>>
> #0  0x00000000600a9257 in ldl_he_p (ptr=0x4c2c061000000000) at
> /home/vfazio/development/qemu/include/qemu/bswap.h:351
> #1  0x00000000600a92fe in ldl_be_p (ptr=0x4c2c061000000000) at
> /home/vfazio/development/qemu/include/qemu/bswap.h:449
> #2  0x00000000600c0790 in translator_ldl_swap (env=0x62923150,
> pc=5488768711647035392, do_swap=false) at
> /home/vfazio/development/qemu/include/exec/translator.h:201
> #3  0x000000006011c1ab in ppc_tr_translate_insn (dcbase=0x7fffffffd250,
> cs=0x6291ae80) at /home/vfazio/development/qemu/target/ppc/translate.c:7856
> #4  0x000000006005ae70 in translator_loop (ops=0x60805fc0 <ppc_tr_ops>,
> db=0x7fffffffd250, cpu=0x6291ae80, tb=0x60a5f900
> <static_code_gen_buffer+1681600>, max_insns=512)
>     at /home/vfazio/development/qemu/accel/tcg/translator.c:102
> <<<<<
> 
> The handler has what looks like an LE address but then swaps it since
> the host is LE and the target is BE
> 
>>>>>>
> setup_rt_frame (sig=4, ka=0x628c5f80 <sigact_table+96>, info=0x62931658,
> set=0x7fffffffd2f8, env=0x62923150) at
> /home/vfazio/development/qemu/linux-user/ppc/signal.c:575
> 575            qemu_log_mask(LOG_GUEST_ERROR, "sa_handler NIP to "
> TARGET_FMT_lx "\n", ka->_sa_handler);
> (gdb) p/x ka->_sa_handler
> $11 = 0x10062c4c
> 579            env->nip = tswapl((target_ulong) ka->_sa_handler);
> (gdb) p/x env->nip
> $12 = 0x4c2c061000000000
> <<<<<
> 
> the memcpy later faults because that's obviously not a valid address
> from which to grab the PPC instructions that need to be translated.
> 
> I'm not familiar with the ELFv2 ABI or PPC assembly in general, so I'm
> not sure what to expect here. Typically ELFv2 was reserved for ppc64le,
> but musl uses ELFv2 for all ppc64 targets. This likely wouldn't be an
> issue in ppc64LE since x86_64 is LE as well and no swap would take place.
> 
> Is the signal handler address tied to the endianess of the host? I
> noticed there was no swapping in elfload.c so wasn't sure...removing the
> swap gets me a little further in program execution but i do eventually
> crash (but that may not be related to this specific issue).

To be sure it's not a problem in the qemu linux-user part, could you run
your binaries on a real hardware or with qemu-system-ppc64?

Or provide me a tar.gz with the binaries for the target, I will test
them on my PowerMac G5 (64bit BE).

Thanks,
Laurent