Re: [Qemu-devel] [Bug 1754542] Re: colo: vm crash with segmentation fault

[Zhang Chen]

Hi Suiheng,

I made a new guest image and retest it, and got the same bug from latest
branch.
I found that after the COLO checkpoint begin, the secondary guest always
send
reset request to Qemu like someone still push the reset button in the guest.
And this bug occurred in COLO frame related codes. This part of codes wrote
by Li zhijian and Zhang hailiang and currently maintained by Zhang hailiang.
So, I add them to this thread.

CC Zhijian and Hailiang:
Any idea or comments about this bug?

If you want to test COLO currently, you can try the old version of COLO:
https://github.com/zhangckid/qemu/tree/qemu-colo-18mar10-legacy


Thanks
Zhang Chen

On Mon, Mar 19, 2018 at 10:08 AM, 李穗恒 <address@hidden> wrote:

> Hi Zhang Chen，
> I follow the https://wiki.qemu.org/Features/COLO, And Vm no crash.
> But SVM rebooting constantly after print RESET, PVM normal startup.
>
> Secondary:
> {"timestamp": {"seconds": 1521421788, "microseconds": 541058}, "event":
> "RESUME"}
> {"timestamp": {"seconds": 1521421808, "microseconds": 493484}, "event":
> "STOP"}
> {"timestamp": {"seconds": 1521421808, "microseconds": 686466}, "event":
> "RESUME"}
> {"timestamp": {"seconds": 1521421808, "microseconds": 696152}, "event":
> "RESET", "data": {"guest": true}}
> {"timestamp": {"seconds": 1521421808, "microseconds": 740653}, "event":
> "RESET", "data": {"guest": true}}
> {"timestamp": {"seconds": 1521421818, "microseconds": 742222}, "event":
> "STOP"}
> {"timestamp": {"seconds": 1521421818, "microseconds": 969883}, "event":
> "RESUME"}
> {"timestamp": {"seconds": 1521421818, "microseconds": 979986}, "event":
> "RESET", "data": {"guest": true}}
> {"timestamp": {"seconds": 1521421819, "microseconds": 22652}, "event":
> "RESET", "data": {"guest": true}}
>
>
> The command(I run two VM in sample machine):
>
> Primary:
> sudo /home/lee/Documents/qemu/x86_64-softmmu/qemu-system-x86_64
> -enable-kvm -boot c -m 2048 -smp 2 -qmp stdio  -name primary -cpu
> qemu64,+kvmclock -device piix3-usb-uhci -device usb-tablet \
>     -netdev 
> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
> -device rtl8139,id=e0,netdev=hn0 \
>     -chardev socket,id=mirror0,host=192.168.0.33,port=9003,server,nowait \
>     -chardev socket,id=compare1,host=192.168.0.33,port=9004,server,wait \
>     -chardev socket,id=compare0,host=192.168.0.33,port=9001,server,nowait
> \
>     -chardev socket,id=compare0-0,host=192.168.0.33,port=9001 \
>     -chardev socket,id=compare_out,host=192.168.0.33,port=9005,server,nowait
> \
>     -chardev socket,id=compare_out0,host=192.168.0.33,port=9005 \
>     -object filter-mirror,id=m0,netdev=hn0,queue=tx,outdev=mirror0 \
>     -object filter-redirector,netdev=hn0,id=redire0,queue=rx,indev=compare_out
> \
>     -object filter-redirector,netdev=hn0,id=redire1,queue=rx,outdev=compare0
> \
>     -object iothread,id=iothread1 \
>     -object colo-compare,id=comp0,primary_in=compare0-0,secondary_in=
> compare1,outdev=compare_out0,iothread=iothread1 \
>     -drive if=ide,id=colo-disk0,driver=quorum,read-pattern=fifo,vote-
> threshold=1,children.0.file.filename=/var/lib/libvirt/
> images/1.raw,children.0.driver=raw -S
>
> Secondary:
> sudo /home/lee/Documents/qemu/x86_64-softmmu/qemu-system-x86_64 -boot c
> -m 2048 -smp 2 -qmp stdio  -name secondary -enable-kvm -cpu
> qemu64,+kvmclock \
>     -device piix3-usb-uhci -device usb-tablet \
>     -netdev 
> tap,id=hn0,vhost=off,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown
> \
>     -device rtl8139,netdev=hn0 \
>     -chardev socket,id=red0,host=192.168.0.33,port=9003,reconnect=1 \
>     -chardev socket,id=red1,host=192.168.0.33,port=9004,reconnect=1 \
>     -object filter-redirector,id=f1,netdev=hn0,queue=tx,indev=red0 \
>     -object filter-redirector,id=f2,netdev=hn0,queue=rx,outdev=red1 \
>     -object filter-rewriter,id=rew0,netdev=hn0,queue=all \
>     -drive if=none,id=colo-disk0,file.filename=/var/lib/libvirt/
> images/2.raw,driver=raw,node-name=node0 \
>     -drive if=ide,id=active-disk0,driver=replication,mode=secondary,
> file.driver=qcow2,top-id=active-disk0,file.file.
> filename=/mnt/ramfs/active_disk.img,file.backing.driver=
> qcow2,file.backing.file.filename=/mnt/ramfs/hidden_
> disk.img,file.backing.backing=colo-disk0 \
>     -incoming tcp:0:8888
>
> Secondary:
>   {'execute':'qmp_capabilities'}
>   { 'execute': 'nbd-server-start',
>     'arguments': {'addr': {'type': 'inet', 'data': {'host':
> '192.168.0.33', 'port': '8889'} } }
>   }
>   {'execute': 'nbd-server-add', 'arguments': {'device': 'colo-disk0',
> 'writable': true } }
>   {'execute': 'trace-event-set-state', 'arguments': {'name': 'colo*',
> 'enable': true} }
>
>
> Primary:
>   {'execute':'qmp_capabilities'}
>   { 'execute': 'human-monitor-command',
>     'arguments': {'command-line': 'drive_add -n buddy
> driver=replication,mode=primary,file.driver=nbd,file.
> host=192.168.0.33,file.port=8889,file.export=colo-disk0,node-name=node0'}}
>   { 'execute':'x-blockdev-change', 'arguments':{'parent': 'colo-disk0',
> 'node': 'node0' } }
>   { 'execute': 'migrate-set-capabilities',
>         'arguments': {'capabilities': [ {'capability': 'x-colo', 'state':
> true } ] } }
>   { 'execute': 'migrate', 'arguments': {'uri': 'tcp:192.168.0.33:8888' } }
>
> Thanks
> Suiheng
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1754542
>
> Title:
>   colo:  vm crash with segmentation fault
>
> Status in QEMU:
>   New
>
> Bug description:
>   I use Arch Linux x86_64
>   Zhang Chen's(https://github.com/zhangckid/qemu/tree/qemu-colo-18mar10)
>   Following document 'COLO-FT.txt',
>   I test colo feature on my hosts
>
>   I run this command
>   Primary:
>   sudo /usr/local/bin/qemu-system-x86_64 -enable-kvm -m 2048 -smp 2 -qmp
> stdio -name primary \
>   -device piix3-usb-uhci \
>   -device usb-tablet -netdev tap,id=hn0,vhost=off \
>   -device virtio-net-pci,id=net-pci0,netdev=hn0 \
>   -drive if=virtio,id=primary-disk0,driver=quorum,read-pattern=
> fifo,vote-threshold=1,\
>   children.0.file.filename=/var/lib/libvirt/images/1.raw,\
>   children.0.driver=raw -S
>
>   Secondary:
>   sudo /usr/local/bin/qemu-system-x86_64 -enable-kvm -m 2048 -smp 2 -qmp
> stdio -name secondary \
>   -device piix3-usb-uhci \
>   -device usb-tablet -netdev tap,id=hn0,vhost=off \
>   -device virtio-net-pci,id=net-pci0,netdev=hn0 \
>   -drive if=none,id=secondary-disk0,file.filename=/var/lib/
> libvirt/images/2.raw,driver=raw,node-name=node0 \
>   -drive if=virtio,id=active-disk0,driver=replication,mode=secondary,\
>   file.driver=qcow2,top-id=active-disk0,\
>   file.file.filename=/mnt/ramfs/active_disk.img,\
>   file.backing.driver=qcow2,\
>   file.backing.file.filename=/mnt/ramfs/hidden_disk.img,\
>   file.backing.backing=secondary-disk0 \
>   -incoming tcp:0:8888
>
>   Secondary:
>   {'execute':'qmp_capabilities'}
>   { 'execute': 'nbd-server-start',
>     'arguments': {'addr': {'type': 'inet', 'data': {'host':
> '192.168.0.34', 'port': '8889'} } }
>   }
>   {'execute': 'nbd-server-add', 'arguments': {'device': 'secondary-disk0',
> 'writable': true } }
>
>   Primary:
>   {'execute':'qmp_capabilities'}
>   { 'execute': 'human-monitor-command',
>     'arguments': {'command-line': 'drive_add -n buddy
> driver=replication,mode=primary,file.driver=nbd,file.
> host=192.168.0.34,file.port=8889,file.export=secondary-
> disk0,node-name=nbd_client0'}}
>   { 'execute':'x-blockdev-change', 'arguments':{'parent': 'primary-disk0',
> 'node': 'nbd_client0' } }
>   { 'execute': 'migrate-set-capabilities',
>         'arguments': {'capabilities': [ {'capability': 'x-colo', 'state':
> true } ] } }
>   { 'execute': 'migrate', 'arguments': {'uri': 'tcp:192.168.0.34:8888' } }
>   And two VM with cash
>   Primary:
>   {"timestamp": {"seconds": 1520763655, "microseconds": 511415}, "event":
> "RESUME"}
>   [1]    329 segmentation fault  sudo /usr/local/bin/qemu-system-x86_64
> -boot c -enable-kvm -m 2048 -smp 2 -qm
>
>   Secondary:
>   {"timestamp": {"seconds": 1520763655, "microseconds": 510907}, "event":
> "RESUME"}
>   [1]    367 segmentation fault  sudo /usr/local/bin/qemu-system-x86_64
> -boot c -enable-kvm -m 2048 -smp 2 -qm
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/qemu/+bug/1754542/+subscriptions
>