[Qemu-devel] analysis of LTP test failures under QEMU linux-user (round 2)

[Peter Maydell]

This is an update of a similar post I made back in June.
This time around (1) I'm using QEMU master because we
got all the necessary fixes into it (2) I'm using an
updated LTP which also has some fixes.

Although it would obviously be nice to fix some of
the remaining issues here, I don't think any of them
are particularly important to real-world programs.


aarch64 and armhf, QEMU git commit 1b0d3845b454 and LTP commit 0119096bd11.

ARM:
Total Tests: 1093
Total Skipped Tests: 80
Total Failures: 57
Kernel Version: 4.4.0-31-generic
Machine Architecture: armv7l

AArch64:
Total Tests: 1093
Total Skipped Tests: 142
Total Failures: 47
Kernel Version: 4.4.0-31-generic
Machine Architecture: aarch64

Unimplemented syscalls:
adjtimex01                    NR_adjtimex not implemented
adjtimex02                    NR_adjtimex not implemented
mq_notify01                   mq_notify syscall not implemented
mq_notify02                   ditto
process_vm_readv01            process_vm_readv syscall not implemented
process_vm_readv02            ditto
process_vm_readv03            ditto
process_vm_writev01           process_vm_writev syscall not implemented
process_vm_writev02           ditto
ptrace01                      ptrace not implemented
ptrace02                      ditto
ptrace03                      ditto
ptrace05                      ditto
remap_file_pages01            remap_file_pages syscall unimplemented
remap_file_pages02            ditto
preadv01                      preadv
preadv01_64
preadv02
preadv02_64
pwritev01                     pwritev
pwritev01_64
pwritev02
pwritev02_64
sched_setattr01               sched_setattr
sched_getattr01               sched_getattr
sched_getattr02               ditto
(Some 'skipped' tests are also skipped because we don't implement
the necessary syscalls; those aren't listed here.)

Unimplemented flags to implemented syscalls
clone05                       CLONE_VFORK not supported
clone08                       oddball clone flags not supported
kcmp03                        ditto
setns02                       we don't support CLONE_NEWUTS, CLONE_NEWIPC
ioctl03                       no TUNGETFEATURES (or other tun/tap support)

Unfixable in QEMU:
creat07                       creat on text's executable doesn't fail ETXTBSY
 -- this is unfixable because the kernel doesn't allow userspace to
    make mappings fail ETXTBSY; only true executables executed by the
    kernel can get there.
futex_wait02                  wants set_robust_list and we can't do that
futex_wake03                  ditto

Wants to operate on signal 64 (which under QEMU is reserved to the host libc):
rt_sigaction01                wants to test signal 64
sighold02                     wants to operate on signal 64
sigrelse01                    wants to operate on signal 64

Test for a bug which is present in my host kernel
writev07

Misc unfixed:
fcntl34                       fcntl F_OFD_SETLKW fails EINVAL
fcntl34_64                    ditto
inotify06                     spurious fail? works fine run individually
mmap12                        test uses /proc/self/pagemap which we
don't emulate
mmap13                        POSIX mandates SIGBUS for outside-mmap accesses,
                              but QEMU ends up squashing all SIGBUS into SIGSEGV
mprotect01                    mprotect(0) should fail ENOMEM
mremap03                      mremap doesn't fail when it should
msync03                       msync should fail ENOMEM on unmapped memory
munlock02                     munlock should fail ENOMEM on unmapped memory
mincore01                     mincore on unmapped memory or overlong length
                              should fail ENOMEM
madvise02                     madvise should fail for misaligned address,
                              bad advice value, etc etc etc
readahead02                   intermittent? "failed to save any I/O"
setgroups03                   libc wrapper for setgroups() will abort() for some
                              error cases but syscall should return an errno
setgroups03_16                ditto
setregid02                    ditto
setregid02_16                 ditto
setreuid06                    ditto
setreuid06_16                 ditto
setxattr03                    only fails on arm, not aarch64

Tests which we skip because they hang or otherwise break things:
clone02                       we aren't implementing clone flags correctly so
                              the child gets the wrong return value for
                              getppid() and kills the test harness by accident
fork13                        hangs
futex_wait03                  hangs
msgctl11                      runs OK but thrashes the machine with lots
                              of processes
syslog*                       these try to restart syslogd, which is a bad
                              plan in a chroot
mmap16                        gets rather confused, and in failing it doesn't
                              clean up a loopback mount it set up, which makes
                              later tests fail

thanks
-- PMM