Re: [Qemu-devel] [PATCH v4 1/4] iotests: add script_initialize

[John Snow]

On 9/18/19 12:44 PM, Vladimir Sementsov-Ogievskiy wrote:
> 18.09.2019 13:30, Vladimir Sementsov-Ogievskiy wrote:
>> 18.09.2019 1:29, John Snow wrote:
>>>
>>>
>>> On 9/16/19 10:56 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> 12.09.2019 3:16, John Snow wrote:
>>>>> Like script_main, but doesn't require a single point of entry.
>>>>> Replace all existing initialization sections with this drop-in 
>>>>> replacement.
>>>>>
>>>>> This brings debug support to all existing script-style iotests.
>>>>>
>>>>> Note: supported_oses=['linux'] was omitted, as it is a default argument.
>>>>
>>>> But after this patch all test which didn't check os start to check linux
>>>> (as it's default).. So all tests which worked on other platforms will now
>>>> be skipped on these other platforms?
>>>>
>>>> Finally do we support something except linux for iotests?
>>>> for bash tests _supported_os also used only with "Linux" in 87 tests..
>>>>
>>>> May be we'd better drop both _supported_os and supported_oses alltogether,
>>>> and don't care?
>>>>
>>>> Anyway, if we support only linux, any reason to skip almost all tests,
>>>> if someone tries to run test on other platform? Let him run what he wants.
>>>>
>>>
>>> Currently, the following tests are python:
>>>
>>> 030 040 041 044 045 055 056 057 065 093 096 118 124 129 132 136 139 147
>>> 148 149 151 152 155 163 165 169 194 196 199 202 203 205 206 207 208 209
>>> 210 211 212 213 216 218 219 222 224 228 234 235 236 237 238 242 245 246
>>> 248 254 255 256 257 258 262 266
>>>
>>> And as it stands, none of the script-style python tests we've selected
>>> to run in `make check` fail on the FreeBSD platform.
>>>
>>> So as an experiment, I lifted the restriction on python tests. I kept
>>> running ./check until I found some invocation that they didn't skip.
>>>
>>> Failures: 045 147 149 169 194 199 211
>>>
>>> Not too bad...
>>>
>>> 045: +qemu.machine.QEMUMachineError: socket_scm_helper does not exist
>>> 149: Wants to use 'sudo', but our freebsd image doesn't have that.
>>> 194: +OSError: AF_UNIX path too long
>>> 211:
>>> -[{ "start": 0, "length": 3072, "depth": 0, "zero": false, "data": true,
>>> "offset": 1024},
>>> -{ "start": 3072, "length": 33551360, "depth": 0, "zero": true, "data":
>>> true, "offset": 4096}]
>>> +[{ "start": 0, "length": 31744, "depth": 0, "zero": false, "data":
>>> true, "offset": 1024},
>>> +{ "start": 31744, "length": 33522688, "depth": 0, "zero": true, "data":
>>> true, "offset": 32768}]
>>>
>>>
>>> 149 can probably be fixed, and 194 and 211 I have fail in similar ways
>>> on my own Linux machine, so that's probably not BSD's fault.
>>>
>>> Interestingly, 169 and 199, bitmap migration related tests, cause a
>>> SIGSEGV in QEMU ...
>>>
>>>
>>> 169:
>>> +EEEE....EEEE........
>>> +WARNING:qemu.machine:qemu received signal 6:
>>> /usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
>>> -chardev
>>> socket,id=mon,path=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/tmprpc0idbx/qemub-26617-monitor.sock
>>> -mon chardev=mon,mode=control -display none -vga none -qtest
>>> unix:path=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/qemub-26617-qtest.sock
>>> -machine accel=qtest -nodefaults -display none -machine accel=qtest
>>> -incoming defer -drive
>>> if=virtio,id=drive0,file=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/disk_b,format=qcow2,cache=writeback
>>>
>>> The common thread in 169 is the +migbitmap trait, which ... makes me a
>>> little nervous, of course!
>>>
>>>
>>> 199:
>>> +WARNING:qemu.machine:qemu received signal 6:
>>> /usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/../../x86_64-softmmu/qemu-system-x86_64
>>> -chardev
>>> socket,id=mon,path=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/tmpvzpyc9j6/qemub-30170-monitor.sock
>>> -mon chardev=mon,mode=control -display none -vga none -qtest
>>> unix:path=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/qemub-30170-qtest.sock
>>> -machine accel=qtest -nodefaults -display none -machine accel=qtest
>>> -drive
>>> if=virtio,id=drive0,file=/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/disk_b,format=qcow2,cache=none
>>> -incoming exec: cat
>>> '/usr/home/qemu/qemu-test.IfsR68/build/tests/qemu-iotests/scratch/mig_fifo'
>>> +E
>>>
>>>
>>> Vladimir, I was able to provoke this error by editing
>>> ./tests/vm/Makefile.include and removing the --snapshot invocation, then
>>> using `make vm-build-freebsd` and finally typing `make vm-ssh-freebsd`
>>> to open up a shell.
>>>
>>> Then the tricks are the usual ones; navigate to iotests directory,
>>> ./check -v -qcow2 169, etc. You'll need to create a build that allows
>>> Python tests to run; do it before you run the snapshot-less build.
>>>
>>>
>>
>> Interesting, I'll try to reproduce.
> 
> Could you provide more detailed steps?
> 
> # make vm-build-freebsd
>      VM-IMAGE freebsd
> ### Downloading install iso ...
> ### Preparing iso and disk image ...
> /root/.cache/qemu-vm/images/freebsd.img.install.iso.xz (1/1)
>    100 %       595,0 MiB / 851,1 MiB = 0,699   153 MiB/s       0:05
> Failed to prepare guest environment
> Traceback (most recent call last):
>    File "/work/src/qemu/master/tests/vm/basevm.py", line 353, in main
>      return vm.build_image(args.image)
>    File "/work/src/qemu/master/tests/vm/freebsd", line 86, in build_image
>      img_tmp, self.size])
>    File "/usr/lib64/python3.7/subprocess.py", line 342, in check_call
>      retcode = call(*popenargs, **kwargs)
>    File "/usr/lib64/python3.7/subprocess.py", line 323, in call
>      with Popen(*popenargs, **kwargs) as p:
>    File "/usr/lib64/python3.7/subprocess.py", line 775, in __init__
>      restore_signals, start_new_session)
>    File "/usr/lib64/python3.7/subprocess.py", line 1522, in _execute_child
>      raise child_exception_type(errno_num, err_msg, err_filename)
> FileNotFoundError: [Errno 2] No such file or directory: 'qemu-img': 'qemu-img'
> make: *** [/work/src/qemu/master/tests/vm/Makefile.include:47: 
> /root/.cache/qemu-vm/images/freebsd.img] Error 2
> # ls qemu-img
> qemu-img
> 
> What it wants? I've never done such cross-builds before..
> 

uhhh, hm...

No, I really just typed what you had done -- `make vm-build-freebsd` on
my Fedora 30 host. I do happen to have qemu-img installed system-wide,
maybe it's a requirement.

I removed the '--snapshot' argument from the Makefile in that folder so
that once the build finished I could SSH in to the image and poke around
by running iotests manually.

I wouldn't worry about it right now, exactly -- it might be a problem
with the migration specification for "exec cat" that is causing an
issue, if I had to guess. I can try to debug it later, but I won't be
able to get to it too soon.