Re: [PULL 00/18] Block patches

[Peter Maydell]

On Mon, 22 Jun 2020 at 16:11, Max Reitz <mreitz@redhat.com> wrote:
>
> The following changes since commit bae31bfa48b9caecee25da3d5333901a126a06b4:
>
>   Merge remote-tracking branch 
> 'remotes/kraxel/tags/audio-20200619-pull-request' into staging (2020-06-19 
> 22:56:59 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-06-22
>
> for you to fetch changes up to 74c55e4142a7bb835c38d3770c74210cbb1e4fab:
>
>   iotests: don't test qcow2.py inside 291 (2020-06-22 16:05:23 +0200)
>
> ----------------------------------------------------------------
> Block patches:
> - Support modifying a LUKS-encrypted image's keyslots
> - iotest fixes
>
> ----------------------------------------------------------------

Hi; I see various iotest failures, different things on
PPC64 Linux, OpenBSD and FreeBSD, and on an AArch32 build
that happens to not have optional crypto libs installed.

On PPC64 Linux, lots of iotests fail like this:

  TEST    iotest-qcow2: 001 [fail]
QEMU          --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../ppc64-softmmu/qemu-system-ppc64"
-nodefaults -display none -accel qtest
QEMU_IMG      -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-io"  --cache
writeback --aio threads -f qcow2
QEMU_NBD      -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/ppc64 gcc1-power7 3.10.0-862.14.4.el7.ppc64
TEST_DIR      -- /home/pm215/qemu/build/all/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.vvBdnkatyZ
SOCKET_SCM_HELPER --
/home/pm215/qemu/build/all/tests/qemu-iotests/socket_scm_helper

--- /home/pm215/qemu/tests/qemu-iotests/001.out 2015-04-08
18:43:24.908449234 +0000
+++ /home/pm215/qemu/build/all/tests/qemu-iotests/001.out.bad
2020-06-23 10:42:29.262626202 +0000
@@ -1,5 +1,6 @@
 QA output created by 001
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+./common.filter: line 128: readarray: -d: invalid option
+readarray: usage: readarray [-n count] [-O origin] [-s count] [-t]
[-u fd] [-C callback] [-c quantum] [array]

 == reading whole image ==
 read 134217728/134217728 bytes at offset 0
  TEST    iotest-qcow2: 002 [fail]
QEMU          --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../ppc64-softmmu/qemu-system-ppc64"
-nodefaults -display none -accel qtest
QEMU_IMG      -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-io"  --cache
writeback --aio threads -f qcow2
QEMU_NBD      -- "/home/pm215/qemu/build/all/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/ppc64 gcc1-power7 3.10.0-862.14.4.el7.ppc64
TEST_DIR      -- /home/pm215/qemu/build/all/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.vvBdnkatyZ
SOCKET_SCM_HELPER --
/home/pm215/qemu/build/all/tests/qemu-iotests/socket_scm_helper

Looks like you're trying to use a readarray option that doesn't
exist (maybe only exists in newer shells?)

iotests failures on aarch32 which happens to not have some
optional crypto lib dependency installed I guess; these
iotests ought to be made to skip if the functionality they're
testing isn't compiled into this QEMU:

  TEST    iotest-qcow2: 293 [fail]
QEMU          --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodefaults -display none -machine virt -accel qtest
QEMU_IMG      --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-io"
 --cache writeback --aio threads -f qcow2
QEMU_NBD      --
"/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- Linux/aarch64 mustang-maydell 4.15.0-101-generic
TEST_DIR      --
/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.tjBWiNDvED
SOCKET_SCM_HELPER --
/home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/socket_scm_helper

--- /home/peter.maydell/qemu/tests/qemu-iotests/293.out 2020-06-23
10:38:50.091867725 +0000
+++ /home/peter.maydell/qemu/build/all-a32/tests/qemu-iotests/293.out.bad
      2020-06-23 10:59:09.027627162 +0000
@@ -1,5 +1,6 @@
 QA output created by 293
 == creating a test image ==
+qemu-img: TEST_DIR/t.IMGFMT: No crypto library supporting PBKDF in
this build: Function not implemented
 Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432

 == test that key 0 opens the image ==
@@ -7,10 +8,15 @@
 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

 == adding a password to slot 4 ==
+qemu-img: Can't amend encryption options - encryption not present
 == adding a password to slot 1 ==
+qemu-img: Can't amend encryption options - encryption not present
 == adding a password to slot 3 ==
+qemu-img: Can't amend encryption options - encryption not present
 == adding a password to slot 2 ==
+qemu-img: Can't amend encryption options - encryption not present
 == erase slot 4 ==
+qemu-img: Can't amend encryption options - encryption not present

 == all secrets should work ==
 read 4096/4096 bytes at offset 0
@@ -23,19 +29,29 @@
 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

 == erase slot 0 and try it ==
-qemu-io: can't open: Invalid password, cannot unlock any keyslot
+qemu-img: Can't amend encryption options - encryption not present
+read 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
[etc]


On OpenBSD: iotest 293 fails trying to use "seq" when it doesn't exist:
  TEST    iotest-qcow2: 293 [fail]
QEMU          --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodef
aults -display none -machine virt -accel qtest
QEMU_IMG      --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -
f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- OpenBSD/amd64 openbsd 6.6
TEST_DIR      -- /home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.o4s3SCLIsx
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/293.out  Tue
Jun 23 10:38:33 2020
+++ /home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/293.out.bad
 Tue Jun 23 11:06:19 2020
@@ -1,6 +1,6 @@
 QA output created by 293
 == creating a test image ==
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33554432
+Formatting 'TEST_DIR/t.IMGFMT'x0IMGFMT encrypt.format=luks
encrypt.key-secret=sec0 encrypt.iter-time=10 cluster_size=65
536 compression_type=zlib size=33554432 compat=1.1 lazy_refcounts=off
refcount_bits=16

 == test that key 0 opens the image ==
 read 4096/4096 bytes at offset 0
@@ -29,25 +29,26 @@
 qemu-io: can't open: Invalid password, cannot unlock any keyslot

 == filling  4 slots with secret 2 ==
+./293: line 111: seq: command not found

 == adding secret 0 ==

 == adding secret 3 (last slot) ==

 == trying to add another slot (should fail) ==
-qemu-img: Can't add a keyslot - all keyslots are in use
+qemu-img: Could not open
'driver=qcow2,file.filename=/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/scratch/t.qco
w2,encrypt.key-secret=sec2': Invalid password, cannot unlock any keyslot

 == all secrets should work again ==
 read 4096/4096 bytes at offset 0
 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 read 4096/4096 bytes at offset 0
 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io: can't open: Invalid password, cannot unlock any keyslot
 read 4096/4096 bytes at offset 0
 4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 4096/4096 bytes at offset 0
-4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)

 == erase all keys of secret 2==
+qemu-img: No keyslots match given (old) password for erase operation
 == erase all keys of secret 1==
 == erase all keys of secret 0==
 == erasing secret3 will fail now since it is the only secret (in 3 slots) ==

Also OpenBSD, iotest 295 failed because the QEMU dumped core:
  TEST    iotest-qcow2: 295 [fail]
QEMU          --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodef
aults -display none -machine virt -accel qtest
QEMU_IMG      --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -
f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- OpenBSD/amd64 openbsd 6.6
TEST_DIR      -- /home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.o4s3SCLIsx
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/295.out  Tue
Jun 23 10:38:33 2020
+++ /home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/295.out.bad
 Tue Jun 23 11:09:01 2020
@@ -4,19 +4,11 @@
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 {"return": {}}
+WARNING:qemu.machine:qemu received signal 9:
/home/qemu/qemu-test.DyTmtb/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64
-display none -vga none -chardev
socket,id=mon,path=/tmp/tmp.o4s3SCLIsx/qemu-36824-monitor.sock -mon
chardev=mon,mode=control -qtest
unix:path=/tmp/tmp.o4s3SCLIsx/qemu-36824-qtest.sock -accel qtest
-nodefaults -display none -machine virt -accel qtest
 {"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 {"return": {}}
-{"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
-{"return": {}}
-Job failed: Invalid password, cannot unlock any keyslot
 {"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 {"return": {}}
-{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
-{"return": {}}
-{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
-{"return": {}}
-{"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
-{"return": {}}
 Job failed: Refusing to overwrite active keyslot 2 - please erase it first
 {"execute": "job-dismiss", "arguments": {"id": "job_add_key"}}
 {"return": {}}
@@ -33,8 +25,37 @@
 {"return": {}}
 {"execute": "job-dismiss", "arguments": {"id": "job_erase_key"}}
 {"return": {}}
-...
+.E.
+======================================================================
+ERROR: testOldPassword (__main__.EncryptionSetupTestCase)
 ----------------------------------------------------------------------
+Traceback (most recent call last):
+  File 
"/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/../../python/qemu/qmp.py",
line 139, in __get_events
+    ret = self.__json_read(only_event=True)
+  File 
"/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/../../python/qemu/qmp.py",
line 99, in __json_read
+    data = self.__sockfile.readline()
+  File "/usr/local/lib/python3.7/socket.py", line 589, in readinto
+    return self._sock.recv_into(b)
+socket.timeout: timed out
+
+During handling of the above exception, another exception occurred:
+
+Traceback (most recent call last):
+  File "295", line 234, in testOldPassword
+    self.addKeyQmp("testdev", new_secret = self.secrets[1])
+  File "295", line 164, in addKeyQmp
+    self.vm.run_job('job_add_key')
+  File "/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/iotests.py",
line 648, in run_job
+    ev = filter_qmp_event(self.events_wait(events, timeout=wait))
+  File 
"/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/../../python/qemu/machine.py",
line 518, in events_wait
+    event = self._qmp.pull_event(wait=timeout)
+  File 
"/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/../../python/qemu/qmp.py",
line 254, in pull_event
+    self.__get_events(wait)
+  File 
"/home/qemu/qemu-test.DyTmtb/src/tests/qemu-iotests/../../python/qemu/qmp.py",
line 141, in __get_events
+    raise QMPTimeoutError("Timeout waiting for event")
+qemu.qmp.QMPTimeoutError: Timeout waiting for event
+
+----------------------------------------------------------------------
 Ran 3 tests

-OK
+FAILED (errors=1)

FreeBSD: lots of iotest failures, looks like some sed or other
command trying to sanitize output is broken on this host OS
(assuming gnu sed or whatever, maybe?)

  TEST    iotest-qcow2: 001 [fail]
QEMU          --
"/home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/../../aarch64-softmmu/qemu-system-aarch64"
-nodefaults -display none -machine virt -accel qtest
QEMU_IMG      --
"/home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/../../qemu-img"
QEMU_IO       --
"/home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/../../qemu-io"
--cache writeback --aio threads -f qcow2
QEMU_NBD      --
"/home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/../../qemu-nbd"
IMGFMT        -- qcow2 (compat=1.1)
IMGPROTO      -- file
PLATFORM      -- FreeBSD/amd64 freebsd 12.1-RELEASE
TEST_DIR      -- /home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/scratch
SOCK_DIR      -- /tmp/tmp.l5ksaQ9N
SOCKET_SCM_HELPER --

--- /home/qemu/qemu-test.RNYtim/src/tests/qemu-iotests/001.out
2020-06-23 10:38:33.000000000 +0000
+++ /home/qemu/qemu-test.RNYtim/build/tests/qemu-iotests/001.out.bad
 2020-06-23 10:56:47.833020000 +0000
@@ -1,5 +1,5 @@
 QA output created by 001
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728
+Formatting 'TEST_DIR/t.IMGFMT'x0IMGFMT cluster_size=65536
compression_type=zlib size=134217728 compat=1.1 lazy_refcounts=off
refcount_bits=16

 == reading whole image ==
 read 134217728/134217728 bytes at offset 0


thanks
-- PMM