[Qemu-commits] [qemu/qemu] 024e89: readthodocs: fully specify a build environment

[Max Filippov]

  Branch: refs/heads/staging-7.2
  Home:   https://github.com/qemu/qemu
  Commit: 024e8945fabdffd85f14fca2864ba8f99fb4a835
      
https://github.com/qemu/qemu/commit/024e8945fabdffd85f14fca2864ba8f99fb4a835
  Author: Alex Bennée <alex.bennee@linaro.org>
  Date:   2024-01-23 (Tue, 23 Jan 2024)

  Changed paths:
    M .readthedocs.yml
    A docs/requirements.txt

  Log Message:
  -----------
  readthodocs: fully specify a build environment

This is now expected by rtd so I've expanded using their example as
22.04 is one of our supported platforms. I tried to work out if there
was an easy way to re-generate a requirements.txt from our
pythondeps.toml but in the end went for the easier solution.

Cc:  <qemu-stable@nongnu.org>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Message-Id: <20231221174200.2693694-1-alex.bennee@linaro.org>
(cherry picked from commit b16a45bc5e0e329a16af8a2e020a6e7044f9afa2)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: b67924a04838ffb766b57ec2e5815704f46fc454
      
https://github.com/qemu/qemu/commit/b67924a04838ffb766b57ec2e5815704f46fc454
  Author: Paolo Bonzini <pbonzini@redhat.com>
  Date:   2024-01-25 (Thu, 25 Jan 2024)

  Changed paths:
    M accel/tcg/cpu-exec.c
    M accel/tcg/tb-maint.c
    M accel/tcg/translate-all.c
    M include/exec/exec-all.h

  Log Message:
  -----------
  accel/tcg: Revert mapping of PCREL translation block to multiple virtual 
addresses

This is causing regressions that have not been analyzed yet.  Revert the
change on stable branches.

Cc: qemu-stable@nongnu.org
Cc: Michael Tokarev <mjt@tls.msk.ru>
Related: https://gitlab.com/qemu-project/qemu/-/issues/2092
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: 06e11b7589bb6a0d8a004949e829bebcd7a00ec5
      
https://github.com/qemu/qemu/commit/06e11b7589bb6a0d8a004949e829bebcd7a00ec5
  Author: Fiona Ebner <f.ebner@proxmox.com>
  Date:   2024-01-25 (Thu, 25 Jan 2024)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block/io: clear BDRV_BLOCK_RECURSE flag after recursing in 
bdrv_co_block_status

Using fleecing backup like in [0] on a qcow2 image (with metadata
preallocation) can lead to the following assertion failure:

> bdrv_co_do_block_status: Assertion `!(ret & BDRV_BLOCK_ZERO)' failed.

In the reproducer [0], it happens because the BDRV_BLOCK_RECURSE flag
will be set by the qcow2 driver, so the caller will recursively check
the file child. Then the BDRV_BLOCK_ZERO set too. Later up the call
chain, in bdrv_co_do_block_status() for the snapshot-access driver,
the assertion failure will happen, because both flags are set.

To fix it, clear the recurse flag after the recursive check was done.

In detail:

> #0  qcow2_co_block_status

Returns 0x45 = BDRV_BLOCK_RECURSE | BDRV_BLOCK_DATA |
BDRV_BLOCK_OFFSET_VALID.

> #1  bdrv_co_do_block_status

Because of the data flag, bdrv_co_do_block_status() will now also set
BDRV_BLOCK_ALLOCATED. Because of the recurse flag,
bdrv_co_do_block_status() for the bdrv_file child will be called,
which returns 0x16 = BDRV_BLOCK_ALLOCATED | BDRV_BLOCK_OFFSET_VALID |
BDRV_BLOCK_ZERO. Now the return value inherits the zero flag.

Returns 0x57 = BDRV_BLOCK_RECURSE | BDRV_BLOCK_DATA |
BDRV_BLOCK_OFFSET_VALID | BDRV_BLOCK_ALLOCATED | BDRV_BLOCK_ZERO.

> #2  bdrv_co_common_block_status_above
> #3  bdrv_co_block_status_above
> #4  bdrv_co_block_status
> #5  cbw_co_snapshot_block_status
> #6  bdrv_co_snapshot_block_status
> #7  snapshot_access_co_block_status
> #8  bdrv_co_do_block_status

Return value is propagated all the way up to here, where the assertion
failure happens, because BDRV_BLOCK_RECURSE and BDRV_BLOCK_ZERO are
both set.

> #9  bdrv_co_common_block_status_above
> #10 bdrv_co_block_status_above
> #11 block_copy_block_status
> #12 block_copy_dirty_clusters
> #13 block_copy_common
> #14 block_copy_async_co_entry
> #15 coroutine_trampoline

[0]:

> #!/bin/bash
> rm /tmp/disk.qcow2
> ./qemu-img create /tmp/disk.qcow2 -o preallocation=metadata -f qcow2 1G
> ./qemu-img create /tmp/fleecing.qcow2 -f qcow2 1G
> ./qemu-img create /tmp/backup.qcow2 -f qcow2 1G
> ./qemu-system-x86_64 --qmp stdio \
> --blockdev 
> qcow2,node-name=node0,file.driver=file,file.filename=/tmp/disk.qcow2 \
> --blockdev 
> qcow2,node-name=node1,file.driver=file,file.filename=/tmp/fleecing.qcow2 \
> --blockdev 
> qcow2,node-name=node2,file.driver=file,file.filename=/tmp/backup.qcow2 \
> <<EOF
> {"execute": "qmp_capabilities"}
> {"execute": "blockdev-add", "arguments": { "driver": "copy-before-write", 
> "file": "node0", "target": "node1", "node-name": "node3" } }
> {"execute": "blockdev-add", "arguments": { "driver": "snapshot-access", 
> "file": "node3", "node-name": "snap0" } }
> {"execute": "blockdev-backup", "arguments": { "device": "snap0", "target": 
> "node1", "sync": "full", "job-id": "backup0" } }
> EOF

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@yandex-team.ru>
Message-id: 20240116154839.401030-1-f.ebner@proxmox.com
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
(cherry picked from commit 8a9be7992426c8920d4178e7dca59306a18c7a3a)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: 415f21c723320af3f694efffb6fc47484cc03fb0
      
https://github.com/qemu/qemu/commit/415f21c723320af3f694efffb6fc47484cc03fb0
  Author: Jason Wang <jasowang@redhat.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M hw/net/virtio-net.c

  Log Message:
  -----------
  virtio-net: correctly copy vnet header when flushing TX

When HASH_REPORT is negotiated, the guest_hdr_len might be larger than
the size of the mergeable rx buffer header. Using
virtio_net_hdr_mrg_rxbuf during the header swap might lead a stack
overflow in this case. Fixing this by using virtio_net_hdr_v1_hash
instead.

Reported-by: Xiao Lei <leixiao.nop@zju.edu.cn>
Cc: Yuri Benditovich <yuri.benditovich@daynix.com>
Cc: qemu-stable@nongnu.org
Cc: Mauro Matteo Cascella <mcascell@redhat.com>
Fixes: CVE-2023-6693
Fixes: e22f0603fb2f ("virtio-net: reference implementation of hash report")
Reviewed-by: Michael Tokarev <mjt@tls.msk.ru>
Signed-off-by: Jason Wang <jasowang@redhat.com>
(cherry picked from commit 2220e8189fb94068dbad333228659fbac819abb0)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: e9eb53c5fed3f60a4ed533d096fd270268f02dc1
      
https://github.com/qemu/qemu/commit/e9eb53c5fed3f60a4ed533d096fd270268f02dc1
  Author: Ari Sundholm <ari@tuxera.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M block/blklogwrites.c

  Log Message:
  -----------
  block/blklogwrites: Fix a bug when logging "write zeroes" operations.

There is a bug in the blklogwrites driver pertaining to logging "write
zeroes" operations, causing log corruption. This can be easily observed
by setting detect-zeroes to something other than "off" for the driver.

The issue is caused by a concurrency bug pertaining to the fact that
"write zeroes" operations have to be logged in two parts: first the log
entry metadata, then the zeroed-out region. While the log entry
metadata is being written by bdrv_co_pwritev(), another operation may
begin in the meanwhile and modify the state of the blklogwrites driver.
This is as intended by the coroutine-driven I/O model in QEMU, of
course.

Unfortunately, this specific scenario is mishandled. A short example:
    1. Initially, in the current operation (#1), the current log sector
number in the driver state is only incremented by the number of sectors
taken by the log entry metadata, after which the log entry metadata is
written. The current operation yields.
    2. Another operation (#2) may start while the log entry metadata is
being written. It uses the current log position as the start offset for
its log entry. This is in the sector right after the operation #1 log
entry metadata, which is bad!
    3. After bdrv_co_pwritev() returns (#1), the current log sector
number is reread from the driver state in order to find out the start
offset for bdrv_co_pwrite_zeroes(). This is an obvious blunder, as the
offset will be the sector right after the (misplaced) operation #2 log
entry, which means that the zeroed-out region begins at the wrong
offset.
    4. As a result of the above, the log is corrupt.

Fix this by only reading the driver metadata once, computing the
offsets and sizes in one go (including the optional zeroed-out region)
and setting the log sector number to the appropriate value for the next
operation in line.

Signed-off-by: Ari Sundholm <ari@tuxera.com>
Cc: qemu-stable@nongnu.org
Message-ID: <20240109184646.1128475-1-megari@gmx.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
(cherry picked from commit a9c8ea95470c27a8a02062b67f9fa6940e828ab6)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: 3b53bfd4c8cd17aa3cc73d479e10c7cb88b6b0ad
      
https://github.com/qemu/qemu/commit/3b53bfd4c8cd17aa3cc73d479e10c7cb88b6b0ad
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests: add filter_qmp_generated_node_ids()

Add a filter function for QMP responses that contain QEMU's
automatically generated node ids. The ids change between runs and must
be masked in the reference output.

The next commit will use this new function.

Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20240118144823.1497953-2-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
(cherry picked from commit da62b507a20510d819bcfbe8f5e573409b954006)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
(Mjt: context fix in tests/qemu-iotests/iotests.py due to
 v7.2.0-939-gbcc6777ad6 "iotests: Filter child node information")


  Commit: 219cea653e38aac87fa48186a793b6c9bd9a314b
      
https://github.com/qemu/qemu/commit/219cea653e38aac87fa48186a793b6c9bd9a314b
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M tests/qemu-iotests/141
    M tests/qemu-iotests/141.out

  Log Message:
  -----------
  iotests: port 141 to Python for reliable QMP testing

The common.qemu bash functions allow tests to interact with the QMP
monitor of a QEMU process. I spent two days trying to update 141 when
the order of the test output changed, but found it would still fail
occassionally because printf() and QMP events race with synchronous QMP
communication.

I gave up and ported 141 to the existing Python API for QMP tests. The
Python API is less affected by the order in which QEMU prints output
because it does not print all QMP traffic by default.

The next commit changes the order in which QMP messages are received.
Make 141 reliable first.

Cc: Hanna Czenczek <hreitz@redhat.com>
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20240118144823.1497953-3-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
(cherry picked from commit 9ee2dd4c22a3639c5462b3fc20df60c005c3de64)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


  Commit: 8ec90598e922a604c222bdbc6289bed7279dced6
      
https://github.com/qemu/qemu/commit/8ec90598e922a604c222bdbc6289bed7279dced6
  Author: Stefan Hajnoczi <stefanha@redhat.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M monitor/qmp.c
    M qapi/qmp-dispatch.c
    M tests/qemu-iotests/060.out
    M tests/qemu-iotests/071.out
    M tests/qemu-iotests/081.out
    M tests/qemu-iotests/087.out
    M tests/qemu-iotests/108.out
    M tests/qemu-iotests/109
    M tests/qemu-iotests/109.out
    M tests/qemu-iotests/117.out
    M tests/qemu-iotests/120.out
    M tests/qemu-iotests/127.out
    M tests/qemu-iotests/140.out
    M tests/qemu-iotests/143.out
    M tests/qemu-iotests/156.out
    M tests/qemu-iotests/176.out
    M tests/qemu-iotests/182.out
    M tests/qemu-iotests/183.out
    M tests/qemu-iotests/184.out
    M tests/qemu-iotests/185
    M tests/qemu-iotests/185.out
    M tests/qemu-iotests/191.out
    M tests/qemu-iotests/195.out
    M tests/qemu-iotests/223.out
    M tests/qemu-iotests/227.out
    M tests/qemu-iotests/247.out
    M tests/qemu-iotests/273.out
    M tests/qemu-iotests/308
    M tests/qemu-iotests/308.out
    M tests/qemu-iotests/tests/qsd-jobs.out

  Log Message:
  -----------
  monitor: only run coroutine commands in qemu_aio_context

monitor_qmp_dispatcher_co() runs in the iohandler AioContext that is not
polled during nested event loops. The coroutine currently reschedules
itself in the main loop's qemu_aio_context AioContext, which is polled
during nested event loops. One known problem is that QMP device-add
calls drain_call_rcu(), which temporarily drops the BQL, leading to all
sorts of havoc like other vCPU threads re-entering device emulation code
while another vCPU thread is waiting in device emulation code with
aio_poll().

Paolo Bonzini suggested running non-coroutine QMP handlers in the
iohandler AioContext. This avoids trouble with nested event loops. His
original idea was to move coroutine rescheduling to
monitor_qmp_dispatch(), but I resorted to moving it to qmp_dispatch()
because we don't know if the QMP handler needs to run in coroutine
context in monitor_qmp_dispatch(). monitor_qmp_dispatch() would have
been nicer since it's associated with the monitor implementation and not
as general as qmp_dispatch(), which is also used by qemu-ga.

A number of qemu-iotests need updated .out files because the order of
QMP events vs QMP responses has changed.

Solves Issue #1933.

Cc: qemu-stable@nongnu.org
Fixes: 7bed89958bfbf40df9ca681cefbdca63abdde39d ("device_core: use 
drain_call_rcu in in qmp_device_add")
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2215192
Buglink: https://bugzilla.redhat.com/show_bug.cgi?id=2214985
Buglink: https://issues.redhat.com/browse/RHEL-17369
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Message-ID: <20240118144823.1497953-4-stefanha@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Tested-by: Fiona Ebner <f.ebner@proxmox.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
(cherry picked from commit effd60c878176bcaf97fa7ce2b12d04bb8ead6f7)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
(Mjt: omit changes to tests missing in 7.2)


  Commit: fa020ef10b1448ad4bb998dbdc5917e942363e30
      
https://github.com/qemu/qemu/commit/fa020ef10b1448ad4bb998dbdc5917e942363e30
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2024-01-26 (Fri, 26 Jan 2024)

  Changed paths:
    M tests/qtest/meson.build

  Log Message:
  -----------
  qtest: bump aspeed_smc-test timeout to 6 minutes

On a loaded system with --enable-debug, this test can take longer than
5 minutes. Raising the timeout to 6 minutes gives greater headroom for
such situations.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
[thuth: Increase the timeout to 6 minutes for very loaded systems]
Signed-off-by: Thomas Huth <thuth@redhat.com>
Message-Id: <20231215070357.10888-11-thuth@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
(cherry picked from commit e8a12fe31f776c60fec993513cd1b1e66c2b8e29)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>
(Mjt: context fixup in tests/qtest/meson.build)


  Commit: ecc23c6e9503380c276a303754b6324fc6d4c054
      
https://github.com/qemu/qemu/commit/ecc23c6e9503380c276a303754b6324fc6d4c054
  Author: Max Filippov <jcmvbkbc@gmail.com>
  Date:   2024-01-27 (Sat, 27 Jan 2024)

  Changed paths:
    M target/xtensa/mmu_helper.c

  Log Message:
  -----------
  target/xtensa: fix OOB TLB entry access

r[id]tlb[01], [iw][id]tlb opcodes use TLB way index passed in a register
by the guest. The host uses 3 bits of the index for ITLB indexing and 4
bits for DTLB, but there's only 7 entries in the ITLB array and 10 in
the DTLB array, so a malicious guest may trigger out-of-bound access to
these arrays.

Change split_tlb_entry_spec return type to bool to indicate whether TLB
way passed to it is valid. Change get_tlb_entry to return NULL in case
invalid TLB way is requested. Add assertion to xtensa_tlb_get_entry that
requested TLB way and entry indices are valid. Add checks to the
[rwi]tlb helpers that requested TLB way is valid and return 0 or do
nothing when it's not.

Cc: qemu-stable@nongnu.org
Fixes: b67ea0cd7441 ("target-xtensa: implement memory protection options")
Signed-off-by: Max Filippov <jcmvbkbc@gmail.com>
Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
Message-id: 20231215120307.545381-1-jcmvbkbc@gmail.com
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
(cherry picked from commit 604927e357c2b292c70826e4ce42574ad126ef32)
Signed-off-by: Michael Tokarev <mjt@tls.msk.ru>


Compare: https://github.com/qemu/qemu/compare/7837d7e31de0...ecc23c6e9503