Re: [Qemu-ppc] [PATCH] spapr-pci: remove io ports workaround

[Alexey Kardashevskiy]

On 12/11/2013 05:47 PM, Alexey Kardashevskiy wrote:
> On 12/10/2013 06:47 PM, Greg Kurz wrote:
>> On Tue, 10 Dec 2013 13:43:05 +1100
>> Alexey Kardashevskiy <address@hidden> wrote:
>>> On 12/10/2013 03:33 AM, Greg Kurz wrote:
>>>> In the past, IO space could not be mapped into the memory address space
>>>> so we introduced a workaround for that. Nowadays it does not look
>>>> necessary so we can remove the workaround and make sPAPR PCI
>>>> configuration simplier.
>>>>
>>>> This workaround has also an evil side effect with virtio devices:
>>>> because all PHBs have their .io region at the same address, the devices
>>>> get mapped in the .io-alias region of every PHB (AKA. mapped multiple
>>>> times). This breaks the ioeventfd feature and causes qemu to abort()
>>>> when running with KVM and asking for more than one PHB:
>>>>
>>>> $ qemu-system-ppc64 -machine type=pseries,accel=kvm -smp 1 -m 4G \
>>>>   -hda /local/greg/images/fedora-be.qcow2 \
>>>>   -device
>>>> virtio-9p-pci,fsdev=fsdev0,mount_tag=share,bus=pci,ioeventfd=on \
>>>> -fsdev local,security_model=none,id=fsdev0,path=$HOME/share1 \ -device
>>>> spapr-pci-host-bridge,index=15 kvm_mem_ioeventfd_add: error adding
>>>> ioeventfd: File exists Aborted
>>>>
>>>> This will prevent to use virtio and VFIO passthrough at the same time,
>>>> since VFIO needs a dedicated PHB to work on ppc.
>>>>
>>>> Signed-off-by: Alexey Kardashevskiy <address@hidden>
>>>
>>>
>>> I have not seen this version yet so please remove me from "SOB". The patch
>>> you replied to was eventually reworked and went to upstream as
>>> 66aab867cedd2a2d81b4d64eff7c3e0f6f272bbf
>>>
>>
>> Hi Alex,
>>
>> I agree you have not seen this version yet... The patch I replied to was my
>> primary source of inspiration and contains these bits, hence the SOB. 
>> Anyway, the SOB is now removed until you decide to add one yourself. :)
>>
>>> This one might be correct too but I want to try this first :)
>>>
>>
>> Well, I hope it is. Please try it.
> 
> 
> Yep. Tried. Looks good, did not break a thing as far as I can tell, even
> VGA works :)
> 
> 
> Signed-off-by: Alexey Kardashevskiy <address@hidden>

Hm. Nack. This fails:

./qemu-system-ppc64 \
 -trace "events=qemu_trace_events" \
 -L "qemu-ppc64-bios/" \
 -nographic \
 -vga "none" \
 -device \
 virtio-blk-pci,id=virtioiblk0,drive=drive0,bootindex=20,ioeventfd=on \
 -drive \
file=virtimg/fc19_16GB.qcow2,if=none,id=drive0,readonly=off,format=qcow2,media=disk,werror=stop,rerror=stop,discard=on
\
 -S \
 -m "2048" \
 -machine "pseries" \
 -enable-kvm




command line: BOOT_IMAGE=/vmlinux-3.10.0-rc7-aik-guest+
root=UUID=27cde746-128e-4528-b4de-44a00d807ea0 ro rd.md=0 rd.lvm=0 rd.dm=0
vconsole.font=latarcyrheb-sun16 vconsole.keymap=us rd.luks=0 console=hvc0 debug
memory layout at init:
  memory_limit : 0000000000000000 (16 MB aligned)
  alloc_bottom : 0000000003400000
  alloc_top    : 0000000030000000
  alloc_top_hi : 0000000080000000
  rmo_top      : 0000000030000000
  ram_top      : 0000000080000000
instantiating rtas at 0x000000002fff0000... done
boot cpu hw idx 0
copying OF device tree...
Building dt strings...
Building dt structure...
Device tree strings 0x0000000003410000 -> 0x0000000003410817
Device tree struct  0x0000000003420000 -> 0x0000000003430000
[Switching to Thread 0x3fff8ca4eee0 (LWP 10370)]

Breakpoint 8, unassigned_mem_accepts (opaque=0x0, addr=0x10080000052,
size=0x1, is_write=0x1)
    at /home/alexey/p/qemu/memory.c:892
892         return false;
Missing separate debuginfos, use: debuginfo-install
glusterfs-api-3.4.0-8.fc19.ppc64 glusterfs-libs-3.4.0-8.fc19.ppc64
gnutls-3.1.16-1.fc19.ppc64 keyutils-libs-1.5.6-1.fc19.ppc64
libgcc-4.8.2-1.fc19.ppc64 libgcrypt-1.5.3-2.fc19.ppc64
libibverbs-1.1.7-3.fc19.ppc64 libiscsi-1.7.0-5.fc19.ppc64
libpng-1.5.13-2.fc19.ppc64 librdmacm-1.0.17-1.fc19.ppc64
libusbx-1.0.16-3.fc19.ppc64 systemd-libs-204-17.fc19.ppc64
usbredir-0.6-2.fc19.ppc64
(gdb) bt
#0  unassigned_mem_accepts (opaque=0x0, addr=0x10080000052, size=0x1,
is_write=0x1)
    at /home/alexey/p/qemu/memory.c:892
#1  0x00000000103cb238 in memory_region_access_valid (mr=0x10b76ec8
<io_mem_unassigned>,
    addr=0x10080000052, size=0x1, is_write=0x1) at
/home/alexey/p/qemu/memory.c:928
#2  0x00000000103cb4bc in memory_region_dispatch_write (mr=0x10b76ec8
<io_mem_unassigned>,
    addr=0x10080000052, data=0x80, size=0x1) at
/home/alexey/p/qemu/memory.c:976
#3  0x00000000103cebec in io_mem_write (mr=0x10b76ec8 <io_mem_unassigned>,
addr=0x10080000052,
    val=0x80, size=0x1) at /home/alexey/p/qemu/memory.c:1748
#4  0x0000000010329b54 in address_space_rw (as=0x10b80cf0
<address_space_memory>,
    addr=0x10080000052, buf=0x3fff8ad9e190 "\200", len=0x1, is_write=0x1)
    at /home/alexey/p/qemu/exec.c:1941
#5  0x000000001032a000 in cpu_physical_memory_rw (addr=0x10080000052,
    buf=0x3fff8ad9e190 "\200", len=0x1, is_write=0x1) at
/home/alexey/p/qemu/exec.c:2010
#6  0x00000000103231c4 in cpu_physical_memory_write (addr=0x10080000052,
buf=0x3fff8ad9e190,
    len=0x1) at /home/alexey/p/qemu/include/exec/cpu-common.h:68
#7  0x000000001032b5b0 in stb_phys (addr=0x10080000052, val=0x80)
    at /home/alexey/p/qemu/exec.c:2506
#8  0x00000000103a0c10 in h_logical_store (cpu=0x3fff8ada0010,
spapr=0x100118ca410,
    opcode=0x40, args=0x3fff8bfc0030) at
/home/alexey/p/qemu/hw/ppc/spapr_hcall.c:564
#9  0x00000000103a140c in spapr_hypercall (cpu=0x3fff8ada0010, opcode=0x40,
args=0x3fff8bfc0030)
    at /home/alexey/p/qemu/hw/ppc/spapr_hcall.c:737
#10 0x000000001041b080 in kvm_arch_handle_exit (cs=0x3fff8ada0010,
run=0x3fff8bfc0000)
    at /home/alexey/p/qemu/target-ppc/kvm.c:1223
#11 0x00000000103c5cbc in kvm_cpu_exec (cpu=0x3fff8ada0010)
    at /home/alexey/p/qemu/kvm-all.c:1726
#12 0x000000001031902c in qemu_kvm_cpu_thread_fn (arg=0x3fff8ada0010)
    at /home/alexey/p/qemu/cpus.c:874
#13 0x00000080bcd0c29c in start_thread (arg=0x3fff8ad9eee0) at
pthread_create.c:310
#14 0x00000080bcb1ddb0 in .__clone ()
    at ../sysdeps/unix/sysv/linux/powerpc/powerpc64/clone.S:111



Without your patch, unassigned_mem_accepts() is not called so it tells us
that IO stopped working after your patch.

Unfortunately I do not have good 3D imagination, do not understand this
memory region well enough to give advice and cannot tell quickly what
exactly is wrong here :)



-- 
Alexey