Re: [Qemu-devel] [PULL 00/52] testing, gdbstub and cputlb fixes

[Alex Bennée]

Peter Maydell <address@hidden> writes:

> On Fri, 7 Jun 2019 at 10:05, Alex Bennée <address@hidden> wrote:
>>
>> The following changes since commit 0d74f3b4277a7ecb0ccb80c865797d11f8e466f5:
>>
>>   Merge remote-tracking branch 
>> 'remotes/vivier2/tags/trivial-branch-pull-request' into staging (2019-06-06 
>> 14:09:14 +0100)
>>
>> are available in the Git repository at:
>>
>>   https://github.com/stsquad/qemu.git 
>> tags/pull-testing-gdbstub-cputlb-070619-1
>>
>> for you to fetch changes up to 611a3db380ef9f670c5c82837c4889f7e19ded10:
>>
>>   gdbstub: Implement qemu physical memory mode (2019-06-07 08:39:03 +0100)
>>
>> ----------------------------------------------------------------
>> Various fixes and updates:
>>
>>   - editor config tweak for shell scripts
>>   - iotest updates (still not default for make check)
>>   - various docker updates
>>   - gcc/ubsan updates for travis
>>   - clean-ups for tests/vm including autoinstall
>>   - semihosting fix for Coverity
>>   - fixes for cputlb in 64-on-32 cases
>>   - gdbstub re-factor + maintainership update
>>
>
> This seems to break the tests/vm testing I do. Eg openbsd:
>
> hackbox mergebuild: seq openbsd
> make: Entering directory '/home/peter.maydell/qemu-openbsd/build'
> python3 -B /home/peter.maydell/qemu-openbsd/tests/vm/openbsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/openbsd.img"
> --force --build-image
> /home/peter.maydell/.cache/qemu-vm/images/openbsd.img
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpcex0zicz/qemu-6023-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpcex0zicz/qemu-6023-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/openbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/openbsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:<<< {'return': {}}
> DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
> {'command-line': 'info usernet'}}
> DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
> TCP[HOST_FORWARD]  13       127.0.0.1 37043       10.0.2.15    22
> 0     0\r\n'}
> console: *** read timeout ***
> console: waiting for: 'boot>'
> console: line buffer:
>
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line 351, in 
> main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/openbsd", line 85,
> in build_image
>     self.console_wait_send("boot>", "set tty com0\n")
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
> 251, in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-openbsd/tests/vm/basevm.py", line
> 213, in console_wait
>     chars = vm.console_socket.recv(1024)
> socket.timeout: timed out
> DEBUG:QMP:>>> {'execute': 'quit'}
> DEBUG:QMP:<<< {'return': {}}
> /home/peter.maydell/qemu-openbsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/openbsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/openbsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-openbsd/build'
>
> ---
> NetBSD:
>
> address@hidden:~$ mergebuild netbsd
> hackbox mergebuild: seq netbsd
> make: Entering directory '/home/peter.maydell/qemu-netbsd/build'
> python3 -B /home/peter.maydell/qemu-netbsd/tests/vm/netbsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/netbsd.img" --force
> --build-image /home/peter.maydell/.cache/qemu-vm/images/netbsd.img
> ### Preparing iso and disk image ...
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:>>> {'execute': 'quit'}
> WARNING:qemu:qemu received signal 6: qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> DEBUG:qemu:Error launching VM
> DEBUG:qemu:Command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpp_t16skr/qemu-13312-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpp_t16skr/qemu-13312-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso'
> DEBUG:qemu:Output: 'RAMBlock "vga.vram" already registered, abort!\n'
> ERROR:root:Failed to launch QEMU, command line:
> ERROR:root:qemu-system-x86_64 -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/netbsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -device VGA -machine
> graphics=off -cdrom
> /home/peter.maydell/.cache/qemu-vm/images/netbsd.img.install.iso
> ERROR:root:Log:
> ERROR:root:RAMBlock "vga.vram" already registered, abort!
>
> ERROR:root:QEMU version >= 2.10 is required
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 351, in main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/netbsd", line 79, in
> build_image
>     "-cdrom", iso
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/basevm.py", line 170, in boot
>     guest.launch()
>   File 
> "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 297, in launch
>     self._launch()
>   File 
> "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 324, in _launch
>     self._post_launch()
>   File 
> "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/__init__.py",
> line 268, in _post_launch
>     self._qmp.accept()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 157, in accept
>     return self.__negotiate_capabilities()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 75, in __negotiate_capabilities
>     resp = self.cmd('qmp_capabilities')
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 191, in cmd
>     return self.cmd_obj(qmp_cmd)
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 174, in cmd_obj
>     resp = self.__json_read()
>   File "/home/peter.maydell/qemu-netbsd/tests/vm/../../python/qemu/qmp.py",
> line 82, in __json_read
>     data = self.__sockfile.readline()
>   File "/usr/lib/python3.6/socket.py", line 586, in readinto
>     return self._sock.recv_into(b)
> ConnectionResetError: [Errno 104] Connection reset by peer
> /home/peter.maydell/qemu-netbsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/netbsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/netbsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-netbsd/build'
>
>
> FreeBSD:
>
> address@hidden:~$ mergebuild freebsd
> hackbox mergebuild: seq freebsd
> make: Entering directory '/home/peter.maydell/qemu-freebsd/build'
> python3 -B /home/peter.maydell/qemu-freebsd/tests/vm/freebsd  --debug
> --image "/home/peter.maydell/.cache/qemu-vm/images/freebsd.img"
> --force --build-image
> /home/peter.maydell/.cache/qemu-vm/images/freebsd.img
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
>   100 %       595.0 MiB / 851.1 MiB = 0.699   117 MiB/s       0:07
> Formatting '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp',
> fmt=qcow2 size=21474836480 cluster_size=65536 lazy_refcounts=off
> refcount_bits=16
> ### Booting installer ...
> DEBUG:root:QEMU args: -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso
> DEBUG:qemu:VM launch command: 'qemu-system-x86_64 -chardev
> socket,id=mon,path=/var/tmp/tmpc1sqg5ms/qemu-23074-monitor.sock -mon
> chardev=mon,mode=control -display none -vga none -machine pc -chardev
> socket,id=console,path=/var/tmp/tmpc1sqg5ms/qemu-23074-console.sock,server,nowait
> -serial chardev:console -nodefaults -m 4G -cpu max -netdev
> user,id=vnet,hostfwd=:127.0.0.1:0-:22 -device
> virtio-net-pci,netdev=vnet -vnc 127.0.0.1:0,to=20 -smp 18 -enable-kvm
> -device VGA -drive
> file=/home/peter.maydell/.cache/qemu-vm/images/freebsd.img.tmp,if=none,id=drive0,cache=writeback
> -device virtio-blk,drive=drive0,bootindex=0 -machine graphics=off
> -cdrom /home/peter.maydell/.cache/qemu-vm/images/freebsd.img.install.iso'
> DEBUG:QMP:>>> {'execute': 'qmp_capabilities'}
> DEBUG:QMP:<<< {'return': {}}
> DEBUG:QMP:>>> {'execute': 'human-monitor-command', 'arguments':
> {'command-line': 'info usernet'}}
> DEBUG:QMP:<<< {'return': 'VLAN -1 (vnet):\r\n  Protocol[State]    FD
> Source Address  Port   Dest. Address  Port RecvQ SendQ\r\n
> TCP[HOST_FORWARD]  13       127.0.0.1 37719       10.0.2.15    22
> 0     0\r\n'}
> console: *** read timeout ***
> console: waiting for: 'Autoboot'
> console: line buffer:
>
>
> Failed to prepare guest environment
> Traceback (most recent call last):
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line 351, in 
> main
>     return vm.build_image(args.image)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 94,
> in build_image
>     self.console_boot_serial()
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/freebsd", line 71,
> in console_boot_serial
>     self.console_wait_send("Autoboot", "3")
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 251, in console_wait_send
>     self.console_wait(wait)
>   File "/home/peter.maydell/qemu-freebsd/tests/vm/basevm.py", line
> 213, in console_wait
>     chars = vm.console_socket.recv(1024)
> socket.timeout: timed out
> DEBUG:QMP:>>> {'execute': 'quit'}
> DEBUG:QMP:<<< {'timestamp': {'seconds': 1559900897, 'microseconds':
> 206270}, 'event': 'NIC_RX_FILTER_CHANGED', 'data': {'path':
> '/machine/peripheral-anon/device[0]/virtio-backend'}}
> DEBUG:QMP:<<< {'return': {}}
> /home/peter.maydell/qemu-freebsd/tests/vm/Makefile.include:47: recipe
> for target '/home/peter.maydell/.cache/qemu-vm/images/freebsd.img'
> failed
> make: *** [/home/peter.maydell/.cache/qemu-vm/images/freebsd.img] Error 2
> make: Leaving directory '/home/peter.maydell/qemu-freebsd/build'

So I have a branch that works:

  
https://github.com/stsquad/qemu/tree/testing/pull-testing-gdbstub-cputlb-070619-1

Which has the following additional patch:

  Subject: [PATCH] tests/vm: favour the locally built QEMU for bootstrapping
  Date: Fri,  7 Jun 2019 19:53:37 +0100
  Message-Id: <address@hidden>

And reverts the patch:

  tests/vm: netbsd autoinstall, using serial console

Which seems to have problems running on hackbox (serial works but we
don't step through the install menus, lots of qqqqqqqqqqqqqqqqqqqqqqq's
in the logs).

Should I respin the PR with 1 new and 1 dropped?

--
Alex Bennée