Re: [Qemu-devel] [PATCH v6 04/12] tests: Add vm test lib

[Fam Zheng]

On Fri, 09/08 22:25, Philippe Mathieu-Daudé wrote:
> Hi Fam, Alex,
> 
> > > 
> > > If this fails it's fairly cryptic:
> > > 
> > > /home/alex/lsrc/qemu/qemu.git/tests/vm/openbsd  --debug   --image 
> > > "tests/vm/openbsd.img" --build-qemu /home/alex/lsrc/qemu/qemu.git
> > > DEBUG:root:Creating archive ./vm-test-fxejnB.tmp/data-2de24.tar for 
> > > src_dir dir: /home/alex/lsrc/qemu/qemu.git
> > > DEBUG:root:QEMU args: -nodefaults -m 2G -cpu host -netdev 
> > > user,id=vnet,hostfwd=:0.0.0.0:0-:22 -device virtio-net-pci,netdev=vnet 
> > > -vnc :0,to=20 -serial file:./vm-test-fxejnB.tmp/serial.out -smp 4 
> > > -enable-kvm -device VGA -drive 
> > > file=tests/vm/openbsd.img,snapshot=on,if=none,id=drive0,cache=writeback 
> > > -device virtio-blk,drive=drive0,bootindex=0 -drive 
> > > file=./vm-test-fxejnB.tmp/data-2de24.tar,if=none,id=data-2de24,cache=writeback,format=raw
> > >  -device virtio-blk,drive=data-2de24,serial=data-2de24,bootindex=1
> > > Failed to prepare guest environment
> > > Traceback (most recent call last):
> > >    File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 260, in 
> > > main
> > >      vm.boot(args.image + ",snapshot=on")
> > >    File "/home/alex/lsrc/qemu/qemu.git/tests/vm/basevm.py", line 184, in 
> > > boot
> > >      guest.launch()
> > >    File "/home/alex/lsrc/qemu/qemu.git/tests/vm/../../scripts/qemu.py", 
> > > line 151, in launch
> > >      self._post_launch()
> > >    File "/home/alex/lsrc/qemu/qemu.git/tests/vm/../../scripts/qemu.py", 
> > > line 135, in _post_launch
> > >      self._qmp.accept()
> > >    File 
> > > "/home/alex/lsrc/qemu/qemu.git/tests/vm/../../scripts/qmp/qmp.py", line 
> > > 147, in accept
> > >      return self.__negotiate_capabilities()
> > >    File 
> > > "/home/alex/lsrc/qemu/qemu.git/tests/vm/../../scripts/qmp/qmp.py", line 
> > > 60, in __negotiate_capabilities
> > >      raise QMPConnectError
> > > QMPConnectError
> > > 
> > > I assume QEMU failed to boot and might have given us a message that
> > > would be useful.
> > 
> > It's a separate issue of qemu.py, I didn't try to fix it here because 
> > currently
> > there are many patches touching that file on the list, there is a high 
> > chance of
> > conflict.
> > 
> > In your case which qemu version are you using? If not master it probably is
> > failing because of the hostfwd syntax.
> 
> very likely.
> 
> one simple way to handle this could be having this script catching
> QMPConnectError and restart with older command line format,

The error reporting definitely needs improving (like saying which version of
QEMU is needed), but I'm not sure about adding fallbacks here and there are
worth the effort, see below.

> 
> but I prefer starting QEMU binding a QMP socket, then initialize the options
> via QMP packets, this should be more portable, this is cleaner.

This is just different, but not necessarily cleaner. When you want to manually
start such a VM, it will be very painful. Nor does "ps | grep | less" etc works.
Command line is simpler and more user friendly in this respect.

> 
> also I think we don't want to run those images with bleeding edge
> qemu-system, we want a stable QEMU to be able to compile the bleeding edge
> codebase inside a working VM.

You are right we are not testing that bleeding edge qemus can run the VM, but we
need the features of the bleeding edge qemu to write simpler test code. I think
this is reasonable for both CI and developers.  So unless there is an
alternative way to do the hostfwd port allocation, I think we can live with this
requirement.

> 
> another note: currently when a VM fails, it closes and what I've left is
> only the last stderr.
> I'd rather see 2 modes:
> - run from some CI when only the final return value is useful, the stderr
> being extra value.
> - in user-dev mode if the build fails I'd like to have access to the
> terminal, eventually some msg like "this build failed, you can connect to
> this VM copy/pasting the following ssh command: ... or hit ^C to gracefully
> shutdown the VM"
> But in this mode I'd rather mount my current workdir via a read-writable
> netshare rather than a one-time temporary read-only tarball, so I could
> modify my repo and just run 'make' inside the VM.

Like the docker tests, this is what DEBUG=1 is for.

Fam