[Qemu-commits] [qemu/qemu] 50ba5b: block/file-posix: Truncate in xfs_write_zeroes()

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 50ba5b2d994853b38fed10e0841b119da0f8b8e5
      
https://github.com/qemu/qemu/commit/50ba5b2d994853b38fed10e0841b119da0f8b8e5
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  block/file-posix: Truncate in xfs_write_zeroes()

XFS_IOC_ZERO_RANGE does not increase the file length:
$ touch foo
$ xfs_io -c 'zero 0 65536' foo
$ stat -c "size=%s, blocks=%b" foo
size=0, blocks=128

We do want writes beyond the EOF to automatically increase the file
length, however.  This is evidenced by the fact that iotest 061 is
broken on XFS since qcow2's check implementation checks for blocks
beyond the EOF.

Reported-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b6c246942b14d3e0dec46a6c5868ed84e7dbea19
      
https://github.com/qemu/qemu/commit/b6c246942b14d3e0dec46a6c5868ed84e7dbea19
  Author: Alberto Garcia <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE

When an L2 table entry points to a compressed cluster the space used
by the data is specified in 512-byte sectors. This size is independent
from BDRV_SECTOR_SIZE and is specific to the qcow2 file format.

The QCOW2_COMPRESSED_SECTOR_SIZE constant defined in this patch makes
this explicit.

Signed-off-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 41ae31e3d7696ec639b8fd6c162846244eeaa511
      
https://github.com/qemu/qemu/commit/41ae31e3d7696ec639b8fd6c162846244eeaa511
  Author: Alberto Garcia <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/io.c
    M qemu-io-cmds.c

  Log Message:
  -----------
  block: Use BDRV_REQUEST_MAX_BYTES instead of BDRV_REQUEST_MAX_SECTORS

There are a few places in which we turn a number of bytes into sectors
in order to compare the result against BDRV_REQUEST_MAX_SECTORS
instead of using BDRV_REQUEST_MAX_BYTES directly.

Signed-off-by: Alberto Garcia <address@hidden>
Reviewed-by: Stefano Garzarella <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9183dd15334f1eee46051bbdafc6ab9fbfc2efd5
      
https://github.com/qemu/qemu/commit/9183dd15334f1eee46051bbdafc6ab9fbfc2efd5
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M qmp.c

  Log Message:
  -----------
  qmp: forbid qmp_cont in RUN_STATE_FINISH_MIGRATE

qmp_cont in RUN_STATE_FINISH_MIGRATE may lead to moving vm to
RUN_STATE_RUNNING, before actual migration finish. So, when migration
thread will try to go to RUN_STATE_POSTMIGRATE, assuming transition
RUN_STATE_FINISH_MIGRATE->RUN_STATE_POSTMIGRATE, it will crash, as
current state is RUN_STATE_RUNNING, and transition
RUN_STATE_RUNNING->RUN_STATE_POSTMIGRATE is forbidden.

Reported-by: Max Reitz <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Dr. David Alan Gilbert <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 832d78caa51e7ac3c17a37f2c0ed9d904e423a32
      
https://github.com/qemu/qemu/commit/832d78caa51e7ac3c17a37f2c0ed9d904e423a32
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/qemu-iotests/169

  Log Message:
  -----------
  iotest: fix 169: do not run qmp_cont in RUN_STATE_FINISH_MIGRATE

qmp_cont fails if vm in RUN_STATE_FINISH_MIGRATE, so let's wait for
final RUN_STATE_POSTMIGRATE. Also, while being here, check qmp_cont
result.

Reported-by: Max Reitz <address@hidden>
Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Tested-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 25349e8250e67e7d126fbf0ad213811b62e7e529
      
https://github.com/qemu/qemu/commit/25349e8250e67e7d126fbf0ad213811b62e7e529
  Author: Klaus Birkelund Jensen <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M hw/block/nvme.c

  Log Message:
  -----------
  nvme: fix copy direction in DMA reads going to CMB

`nvme_dma_read_prp` erronously used `qemu_iovec_*to*_buf` instead of
`qemu_iovec_*from*_buf` when the request involved the controller memory
buffer.

Signed-off-by: Klaus Birkelund Jensen <address@hidden>
Reviewed-by: Kenneth Heitke <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 5d2318499fe980542fbc21d9ccbc127a6106017b
      
https://github.com/qemu/qemu/commit/5d2318499fe980542fbc21d9ccbc127a6106017b
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Add bdrv_try_set_aio_context()

Eventually, we want to make sure that all parents and all children of a
node are in the same AioContext as the node itself. This means that
changing the AioContext may fail because one of the other involved
parties (e.g. a guest device that was configured with an iothread)
cannot allow switching to a different AioContext.

Introduce a set of functions that allow to first check whether all
involved nodes can switch to a new context and only then do the actual
switch. The check recursively covers children and parents.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a3a683c33d38fb29c7a78903e88dda12b84cc88d
      
https://github.com/qemu/qemu/commit/a3a683c33d38fb29c7a78903e88dda12b84cc88d
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Make bdrv_attach/detach_aio_context() static

Since commit b97511c7bc8, there is no reason for block drivers any more
to call these functions (see the function comment in block_int.h). They
are now just internal helper functions for bdrv_set_aio_context()
and can be made static.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0d83708a1d2effc5d905cd1d61d0ef47d310ad88
      
https://github.com/qemu/qemu/commit/0d83708a1d2effc5d905cd1d61d0ef47d310ad88
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Move recursion to bdrv_set_aio_context()

Instead of having two recursions, in bdrv_attach_aio_context() and in
bdrv_detach_aio_context(), just having one recursion is enough. Said
functions are only about a single node now.

It is important that the recursion doesn't happen between detaching and
attaching a context to the current node because the nested call will
drain the node, and draining with a NULL context would segfault.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 53a7d041858a6787a43012fd04ccf0258389a95d
      
https://github.com/qemu/qemu/commit/53a7d041858a6787a43012fd04ccf0258389a95d
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c
    M include/block/block.h
    M include/block/block_int.h

  Log Message:
  -----------
  block: Propagate AioContext change to parents

All block nodes and users in any connected component of the block graph
must be in the same AioContext, so changing the AioContext of one node
must not only change all of its children, but all of its parents (and
in turn their children etc.) as well.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 7e2f096a823f75ca3f1df1811c406b46254587fd
      
https://github.com/qemu/qemu/commit/7e2f096a823f75ca3f1df1811c406b46254587fd
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/test-block-iothread.c

  Log Message:
  -----------
  test-block-iothread: Test AioContext propagation through the tree

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 38475269d4d9efcc5f1a5aae4834d165248ea13d
      
https://github.com/qemu/qemu/commit/38475269d4d9efcc5f1a5aae4834d165248ea13d
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/block-backend.c

  Log Message:
  -----------
  block: Implement .(can_)set_aio_ctx for BlockBackend

bdrv_try_set_aio_context() currently fails if a BlockBackend is attached
to a node because it doesn't implement the BdrvChildRole callbacks for
AioContext management.

We can allow changing the AioContext of monitor-owned BlockBackends as
long as no device is attached to them.

When setting the AioContext of the root node of a BlockBackend, we now
need to pass blk->root as an ignored child because we don't want the
root node to recursively call back into BlockBackend and execute
blk_do_set_aio_context() a second time.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 980b0f943aad9240f276d7e48e2cf92ae4eb61ca
      
https://github.com/qemu/qemu/commit/980b0f943aad9240f276d7e48e2cf92ae4eb61ca
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/block-backend.c
    M include/sysemu/block-backend.h

  Log Message:
  -----------
  block: Add blk_set_allow_aio_context_change()

Some users (like block jobs) can tolerate an AioContext change for their
BlockBackend. Add a function that tells the BlockBackend that it can
allow changes.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9ff7f0df8757c1c6ca582935c27db377a1eaeb22
      
https://github.com/qemu/qemu/commit/9ff7f0df8757c1c6ca582935c27db377a1eaeb22
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/backup.c
    M block/mirror.c
    M blockjob.c

  Log Message:
  -----------
  blockjob: Propagate AioContext change to all job nodes

Block jobs require that all of the nodes the job is using are in the
same AioContext. Therefore all BdrvChild objects of the job propagate
.(can_)set_aio_context to all other job nodes, so that the switch is
checked and performed consistently even if both nodes are in different
subtrees.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 657e1203203ccb48cb041ecb3f1799bec8349744
      
https://github.com/qemu/qemu/commit/657e1203203ccb48cb041ecb3f1799bec8349744
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M blockjob.c

  Log Message:
  -----------
  blockjob: Remove AioContext notifiers

The notifiers made sure that the job is quiesced and that the
job->aio_context field is updated. The first part is unnecessary today
since bdrv_set_aio_context_ignore() drains the block node, and this
means drainig the block job, too. The second part can be done in the
.set_aio_ctx callback of the block job BdrvChildRole.

The notifiers were problematic because they poll the AioContext while
the graph is in an inconsistent state with some nodes already in the new
context, but others still in the old context. So removing the notifiers
not only simplifies the code, but actually makes the code safer.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 012056f48d2669685e8695561602f852b0d34ff0
      
https://github.com/qemu/qemu/commit/012056f48d2669685e8695561602f852b0d34ff0
  Author: Kevin Wolf <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/test-block-iothread.c

  Log Message:
  -----------
  test-block-iothread: Test AioContext propagation for block jobs

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9c3db310ff0b7473272ae8dce5e04e2f8a825390
      
https://github.com/qemu/qemu/commit/9c3db310ff0b7473272ae8dce5e04e2f8a825390
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  block/file-posix: Unaligned O_DIRECT block-status

Currently, qemu crashes whenever someone queries the block status of an
unaligned image tail of an O_DIRECT image:
$ echo > foo
$ qemu-img map --image-opts driver=file,filename=foo,cache.direct=on
Offset          Length          Mapped to       File
qemu-img: block/io.c:2093: bdrv_co_block_status: Assertion `*pnum &&
QEMU_IS_ALIGNED(*pnum, align) && align > offset - aligned_offset'
failed.

This is because bdrv_co_block_status() checks that the result returned
by the driver's implementation is aligned to the request_alignment, but
file-posix can fail to do so, which is actually mentioned in a comment
there: "[...] possibly including a partial sector at EOF".

Fix this by rounding up those partial sectors.

There are two possible alternative fixes:
(1) We could refuse to open unaligned image files with O_DIRECT
    altogether.  That sounds reasonable until you realize that qcow2
    does necessarily not fill up its metadata clusters, and that nobody
    runs qemu-img create with O_DIRECT.  Therefore, unpreallocated qcow2
    files usually have an unaligned image tail.

(2) bdrv_co_block_status() could ignore unaligned tails.  It actually
    throws away everything past the EOF already, so that sounds
    reasonable.
    Unfortunately, the block layer knows file lengths only with a
    granularity of BDRV_SECTOR_SIZE, so bdrv_co_block_status() usually
    would have to guess whether its file length information is inexact
    or whether the driver is broken.

Fixing what raw_co_block_status() returns is the safest thing to do.

There seems to be no other block driver that sets request_alignment and
does not make sure that it always returns aligned values.

Cc: address@hidden
Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2fab30c80b33cdc6157c7efe6207e54b6835cf92
      
https://github.com/qemu/qemu/commit/2fab30c80b33cdc6157c7efe6207e54b6835cf92
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/qemu-iotests/221
    A tests/qemu-iotests/253
    A tests/qemu-iotests/253.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Test unaligned raw images with O_DIRECT

We already have 221 for accesses through the page cache, but it is
better to create a new file for O_DIRECT instead of integrating those
test cases into 221.  This way, we can make use of
_supported_cache_modes (and _default_cache_mode) so the test is
automatically skipped on filesystems that do not support O_DIRECT.

As part of the split, add _supported_cache_modes to 221.  With that, it
no longer fails when run with -c none or -c directsync.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 987402c5c2c777cf3a1faf26f9a5a82f587bd3a4
      
https://github.com/qemu/qemu/commit/987402c5c2c777cf3a1faf26f9a5a82f587bd3a4
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M qemu-img.texi

  Log Message:
  -----------
  qemu-img.texi: Be specific about JSON object types

Just writing that --output=json outputs JSON information does not really
help; we should also make a note of what QAPI type the result object
has.  (The map subcommand does not emit a QAPI-typed object, but its
section already describes the object structure well enough.)

Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 4db4390ef31648be28626dff51e87e3647485a46
      
https://github.com/qemu/qemu/commit/4db4390ef31648be28626dff51e87e3647485a46
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M qemu-img.texi

  Log Message:
  -----------
  qemu-img.texi: Describe human-readable info output

Ideally, it should be self-explanatory.  However, keys like "disk size"
arguably really are not self-explanatory.  In any case, there is no harm
in going into a some more detail here.

Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 481e0eeef4fdf7e2ed42425e38d0a30ffd0e9b54
      
https://github.com/qemu/qemu/commit/481e0eeef4fdf7e2ed42425e38d0a30ffd0e9b54
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Improve "Block node is read-only" message

This message does not make any sense when it appears as the response to
making an R/W node read-only.  We should detect that case and emit a
different message, then.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a93a42bd918fdbc3e64f6b88c87ccbd6cc09c971
      
https://github.com/qemu/qemu/commit/a93a42bd918fdbc3e64f6b88c87ccbd6cc09c971
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: Let assert_qmp() accept an array

Sometimes we cannot tell which error message qemu will emit, and we do
not care.  With this change, we can then just pass an array of all
possible messages to assert_qmp() and it will choose the right one.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 86a4f599a67b9b709109c7a7c8b7eb91d21c21fd
      
https://github.com/qemu/qemu/commit/86a4f599a67b9b709109c7a7c8b7eb91d21c21fd
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/qemu-iotests/iotests.py

  Log Message:
  -----------
  iotests.py: Fix VM.run_job

log() is in the current module, there is no need to prefix it.  In fact,
doing so may make VM.run_job() unusable in tests that never use
iotests.log() themselves.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: c423a6af592cf36b4f149c54e2966dd0016b7e96
      
https://github.com/qemu/qemu/commit/c423a6af592cf36b4f149c54e2966dd0016b7e96
  Author: Max Reitz <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M tests/qemu-iotests/245
    M tests/qemu-iotests/245.out

  Log Message:
  -----------
  iotests: Make 245 faster and more reliable

Sometimes, 245 fails for me because some stream job has already finished
while the test expects it to still be active.  (With -c none, it fails
basically every time.) The most reliable way to fix this is to simply
set auto_finalize=false so the job will remain in the block graph as
long as we need it.  This allows us to drop the rate limiting, too,
which makes the test faster.

The only problem with this is that there is a single place that yields a
different error message depending on whether the stream job is still
copying data (so COR is enabled) or not (COR has been disabled, but the
job still has the WRITE_UNCHANGED permission on the target node).  We
can easily address that by expecting either error message.

Note that we do not need auto_finalize=false (or rate limiting) for the
active commit job, because It never completes without an explicit
block-job-complete anyway.

Signed-off-by: Max Reitz <address@hidden>
Reviewed-by: Alberto Garcia <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2259637b95bef3116cc262459271de08e038cc66
      
https://github.com/qemu/qemu/commit/2259637b95bef3116cc262459271de08e038cc66
  Author: Peter Maydell <address@hidden>
  Date:   2019-05-20 (Mon, 20 May 2019)

  Changed paths:
    M block.c
    M block/backup.c
    M block/block-backend.c
    M block/file-posix.c
    M block/io.c
    M block/mirror.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h
    M blockjob.c
    M hw/block/nvme.c
    M include/block/block.h
    M include/block/block_int.h
    M include/sysemu/block-backend.h
    M qemu-img.texi
    M qemu-io-cmds.c
    M qmp.c
    M tests/qemu-iotests/169
    M tests/qemu-iotests/221
    M tests/qemu-iotests/245
    M tests/qemu-iotests/245.out
    A tests/qemu-iotests/253
    A tests/qemu-iotests/253.out
    M tests/qemu-iotests/group
    M tests/qemu-iotests/iotests.py
    M tests/test-block-iothread.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- block: AioContext management, part 1
- qmp: forbid qmp_cont in RUN_STATE_FINISH_MIGRATE
- nvme: fix copy direction in DMA reads going to CMB
- file-posix: Fix block status for unaligned raw images with O_DIRECT
- file-posix: Fix xfs_write_zeroes() after EOF
- Documentation and iotests improvements

# gpg: Signature made Mon 20 May 2019 16:12:38 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>" [full]
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (24 commits)
  iotests: Make 245 faster and more reliable
  iotests.py: Fix VM.run_job
  iotests.py: Let assert_qmp() accept an array
  block: Improve "Block node is read-only" message
  qemu-img.texi: Describe human-readable info output
  qemu-img.texi: Be specific about JSON object types
  iotests: Test unaligned raw images with O_DIRECT
  block/file-posix: Unaligned O_DIRECT block-status
  test-block-iothread: Test AioContext propagation for block jobs
  blockjob: Remove AioContext notifiers
  blockjob: Propagate AioContext change to all job nodes
  block: Add blk_set_allow_aio_context_change()
  block: Implement .(can_)set_aio_ctx for BlockBackend
  test-block-iothread: Test AioContext propagation through the tree
  block: Propagate AioContext change to parents
  block: Move recursion to bdrv_set_aio_context()
  block: Make bdrv_attach/detach_aio_context() static
  block: Add bdrv_try_set_aio_context()
  nvme: fix copy direction in DMA reads going to CMB
  iotest: fix 169: do not run qmp_cont in RUN_STATE_FINISH_MIGRATE
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/6d8e75d41c58...2259637b95be