Re: [Qemu-devel] live migration between different qemu versions

[Dr. David Alan Gilbert]

* Alexey (address@hidden) wrote:
> Hello,
> 
> On 2016-01-12 12:19, Dr. David Alan Gilbert wrote:
> >* Alexey (address@hidden) wrote:
> >>Hi David.
> >>
> >>On 2016-01-11 22:51, Dr. David Alan Gilbert wrote:
> >>>* Alexey (address@hidden) wrote:
> >>>>Hello.
> >>>
> >>>Hi,
> >>>
> >>>>I have two servers between which I need have live migration.
> >>>>
> >>>>First server have QEMU emulator version 2.3.0
> >>>>Second server have QEMU emulator version 2.5.0
> >>>>
> >>>>Migration command look like this:
> >>>>/usr/bin/virsh migrate --live DOMAIN_NAME --migrateuri
> >>>>tcp://second.server
> >>>>qemu+ssh://second.server/system --verbose
> >>>>
> >>>>I got next error on console:
> >>>>error: operation failed: migration job: unexpectedly failed
> >>>>
> >>>>And next error in log on second server:
> >>>>2016-01-11T15:07:12.888647Z qemu-system-x86_64: Length too large:
> >>>>/address@hidden/efi-virtio.rom: 0x32000 > 0x2d000: Invalid argument
> >>>>2016-01-11T15:07:12.888685Z qemu-system-x86_64: error while loading
> >>>>state
> >>>>for instance 0x0 of device 'ram'
> >>>>2016-01-11T15:07:12.889733Z qemu-system-x86_64: load of migration
> >>>>failed:
> >>>>Invalid argument
> >>>>
> >>>>In my libvirt domain I have option <rom bar="off"/>.
> >>>>
> >>>>I need to do a live migration in any possible way. How can I do this
> >>>>one?
> >>
> >>>Is the efi-virtio ROM that you're using on the two versions the same
> >>>identical size?
> >
> >>No, I have different efi-virtio.rom files, and I try to find a way which
> >>allows me not to copy the same efi-virtio.rom to all servers.
> >
> >Right, the error does look like it's just complaining about the ROM size:
> >
> >  2016-01-11T15:07:12.888647Z qemu-system-x86_64: Length too large:
> >  /address@hidden/efi-virtio.rom: 0x32000 > 0x2d000: Invalid argument
> >
> >It copes with small variations, but as soon as it flips over a 4K (?)
> >boundary then it has nowhere to put the extra data coming from the
> >larger ROM.
> old server with qemu 2.3.0:
> -rw-r--r-- 1 root root 203264 Nov 21  2014 /usr/lib/ipxe/qemu/efi-virtio.rom
> 
> new server with qemu 2.5.0:
> -rw-r--r-- 1 root root 184320 Oct 19  2014 /usr/lib/ipxe/qemu/efi-virtio.rom
> 
> See, that destination server has smaller rom than source.
> I interested in way which I could do live migration between different ROMs.

You can't do it if the ROMs dont approximately match in size;  what happens here
is that the 2.3.0 machine allocates just over 203k for the ROM (203264=0x31A00
and it's rounded that upto 0x32000).
On the destination it's size is based on the 184320 (=0x2d000), so it hasn't
got room to receive the larger ROM.
One way to fix this is to pad the ROMs; so just adding blank space to the end
of the newer efi-virtio.rom to make it the same size as the old one.

> Also I interested in migration in both side: from old to new and from new to
> old.

OK, but note that backwards migration isn't tested much in
upstream qemu.

> >>>Also, can you post the commandline that you're using please.
> >>I already done this in my first email: virsh migrate...
> >>>>/usr/bin/virsh migrate --live DOMAIN_NAME --migrateuri
> >>>>tcp://second.server
> >>>>qemu+ssh://second.server/system --verbose
> >Do you mean running domain command line?
> >I meant for the QEMU itself; but if we've already found the ROMs are
> >different sizes, lets fix that first and see how you go.
> Do you mean command line of running domain?
> 
> qemu-system-x86_64 -enable-kvm -name 208664-73 -S -machine
> pc-i440fx-1.7,accel=kvm,usb=off -cpu
> kvm64,+lahf_lm,+popcnt,+sse4.2,+sse4.1,+ssse3,+ss,-mca,-mce -m 1024
> -realtime mlock=off -smp 4,sockets=4,cores=1,threads=1 -uuid
> 08cf6bf2-a701-4a00-a0cb-4ecbe514c3d7 -no-user-config -nodefaults -chardev 
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/208664-73.monitor,server,nowait
> -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown
> -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device
> virtio-scsi-pci,id=scsi0,num_queues=4,bus=pci.0,addr=0x4 -device
> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive 
> file=/dev/disk/vbd/208664-73,if=none,id=drive-scsi0-0-0-0,format=raw,cache=none,discard=unmap,aio=native
> -device 
> scsi-hd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=2
> -netdev tap,fd=53,id=hostnet0,vhost=on,vhostfd=54 -device 
> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:00:40:b9,bus=pci.0,addr=0x3,rombar=0
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 -chardev 
> socket,id=charchannel0,path=/var/lib/libvirt/qemu/208664-73.agent,server,nowait
> -device 
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
> -device usb-mouse,id=input0 -device usb-kbd,id=input1 -vnc [::]:12,password
> -device VGA,id=video0,vgamem_mb=16,bus=pci.0,addr=0x2 -device
> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5 -object
> rng-random,id=objrng0,filename=/dev/random -device 
> virtio-rng-pci,rng=objrng0,id=rng0,max-bytes=1024,period=2000,bus=pci.0,addr=0x7
> -msg timestamp=on

OK, nothing too unusual there.

Dave

> >
> >Dave
> >
> >>
> >>Regards,
> >>Alexey Mochkin.
> >--
> >Dr. David Alan Gilbert / address@hidden / Manchester, UK
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK