[Qemu-commits] [qemu/qemu] 765d9d: block: Typo fix in copy_on_readv()

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 765d9df9626f45a821f221f7a46ef524354b3600
      
https://github.com/qemu/qemu/commit/765d9df9626f45a821f221f7a46ef524354b3600
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Typo fix in copy_on_readv()

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: a8b42a1c09e751b9f921a1a73756411fc118020b
      
https://github.com/qemu/qemu/commit/a8b42a1c09e751b9f921a1a73756411fc118020b
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c

  Log Message:
  -----------
  block: Make bdrv_img_create() size selection easier to read

All callers of bdrv_img_create() pass in a size, or -1 to read the
size from the backing file.  We then set that size as the QemuOpt
default, which means we will reuse that default rather than the
final parameter to qemu_opt_get_size() several lines later.  But
it is rather confusing to read subsequent checks of 'size == -1'
when it looks (without seeing the full context) like size defaults
to 0; it also doesn't help that a size of 0 is valid (for some
formats).

Rework the logic to make things more legible.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ecbfa2817d61fd102d291487ec1e61571b3b6581
      
https://github.com/qemu/qemu/commit/ecbfa2817d61fd102d291487ec1e61571b3b6581
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M include/qemu/hbitmap.h
    M tests/test-hbitmap.c
    M util/hbitmap.c

  Log Message:
  -----------
  hbitmap: Rename serialization_granularity to serialization_align

The only client of hbitmap_serialization_granularity() is dirty-bitmap's
bdrv_dirty_bitmap_serialization_align().  Keeping the two names consistent
is worthwhile, and the shorter name is more representative of what the
function returns (the required alignment to be used for start/count of
other serialization functions, where violating the alignment causes
assertion failures).

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 113754f3a83dac7989ca94c4408a494560df1d73
      
https://github.com/qemu/qemu/commit/113754f3a83dac7989ca94c4408a494560df1d73
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2-bitmap.c

  Log Message:
  -----------
  qcow2: Ensure bitmap serialization is aligned

When subdividing a bitmap serialization, the code in hbitmap.c
enforces that start/count parameters are aligned (except that
count can end early at end-of-bitmap).  We exposed this required
alignment through bdrv_dirty_bitmap_serialization_align(), but
forgot to actually check that we comply with it.

Fortunately, qcow2 is never dividing bitmap serialization smaller
than one cluster (which is a minimum of 512 bytes); so we are
always compliant with the serialization alignment (which insists
that we partition at least 64 bits per chunk) because we are doing
at least 4k bits per chunk.

Still, it's safer to add an assertion (for the unlikely case that
we'd ever support a cluster smaller than 512 bytes, or if the
hbitmap implementation changes what it considers to be aligned),
rather than leaving bdrv_dirty_bitmap_serialization_align()
without a caller.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: dfe55c35772b7ebc6f282a94d4048c9f85b7b26b
      
https://github.com/qemu/qemu/commit/dfe55c35772b7ebc6f282a94d4048c9f85b7b26b
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M include/block/dirty-bitmap.h

  Log Message:
  -----------
  dirty-bitmap: Drop unused functions

We had several functions that no one is currently using, and which
use sector-based interfaces.  I'm trying to convert towards byte-based
interfaces, so it's easier to just drop the unused functions:

bdrv_dirty_bitmap_get_meta
bdrv_dirty_bitmap_get_meta_locked
bdrv_dirty_bitmap_reset_meta
bdrv_dirty_bitmap_meta_granularity

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1b6cc579ded4f9d7923d9d59147512edd623f4c0
      
https://github.com/qemu/qemu/commit/1b6cc579ded4f9d7923d9d59147512edd623f4c0
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c
    M block/dirty-bitmap.c
    M include/block/dirty-bitmap.h

  Log Message:
  -----------
  dirty-bitmap: Avoid size query failure during truncate

We've previously fixed several places where we failed to account
for possible errors from bdrv_nb_sectors().  Fix another one by
making bdrv_dirty_bitmap_truncate() take the new size from the
caller instead of querying itself; then adjust the sole caller
bdrv_truncate() to pass the size just determined by a successful
resize, or to reuse the size given to the original truncate
operation when refresh_total_sectors() was not able to confirm the
actual size (the two sizes can potentially differ according to
rounding constraints), thus avoiding sizing the bitmaps to -1.
This also fixes a bug where not all failure paths in
bdrv_truncate() would set errp.

Note that bdrv_truncate() is still a bit awkward.  We may want
to revisit it later and clean up things to better guarantee that
a resize attempt either fails cleanly up front, or cannot fail
after guest-visible changes have been made (if temporary changes
are made, then they need to be cleanly rolled back).  But that
is a task for another day; for now, the goal is the bare minimum
fix to ensure that just bdrv_dirty_bitmap_truncate() cannot fail.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ebfcd2e75f719c5d74ba72bbca84fa9854b6698f
      
https://github.com/qemu/qemu/commit/ebfcd2e75f719c5d74ba72bbca84fa9854b6698f
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/qcow2-bitmap.c

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_dirty_bitmap_size() to report bytes

We're already reporting bytes for bdrv_dirty_bitmap_granularity();
mixing bytes and sectors in our return values is a recipe for
confusion.  A later cleanup will convert dirty bitmap internals
to be entirely byte-based, but in the meantime, we should report
the bitmap size in bytes.

The only external caller in qcow2-bitmap.c is temporarily more verbose
(because it is still using sector-based math), but will later be
switched to track progress by bytes instead of sectors.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 993e6525bfcc67ba48fe55bd64ec043a4b721e1d
      
https://github.com/qemu/qemu/commit/993e6525bfcc67ba48fe55bd64ec043a4b721e1d
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c

  Log Message:
  -----------
  dirty-bitmap: Track bitmap size by bytes

We are still using an internal hbitmap that tracks a size in sectors,
with the granularity scaled down accordingly, because it lets us
use a shortcut for our iterators which are currently sector-based.
But there's no reason we can't track the dirty bitmap size in bytes,
since it is (mostly) an internal-only variable (remember, the size
is how many bytes are covered by the bitmap, not how many bytes the
bitmap occupies).  A later cleanup will convert dirty bitmap
internals to be entirely byte-based, eliminating the intermediate
sector rounding added here; and technically, since bdrv_getlength()
already rounds up to sectors, our use of DIV_ROUND_UP is more for
theoretical completeness than for any actual rounding.

Use is_power_of_2() while at it, instead of open-coding that.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 86f6ae67e157362f3b141649874213ce01dcc622
      
https://github.com/qemu/qemu/commit/86f6ae67e157362f3b141649874213ce01dcc622
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/qcow2-bitmap.c
    M include/block/dirty-bitmap.h

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_dirty_bitmap_*serialize*() to take bytes

Right now, the dirty-bitmap code exposes the fact that we use
a scale of sector granularity in the underlying hbitmap to anything
that wants to serialize a dirty bitmap.  It's nicer to uniformly
expose bytes as our dirty-bitmap interface, matching the previous
change to bitmap size.  The only caller to serialization is currently
qcow2-cluster.c, which becomes a bit more verbose because it is still
tracking sectors for other reasons, but a later patch will fix that
to more uniformly use byte offsets everywhere.  Likewise, within
dirty-bitmap, we have to add more assertions that we are not
truncating incorrectly, which can go away once the internal hbitmap
is byte-based rather than sector-based.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: c7e7c87ac8e03fe088f4b1f820da30e85a07fd6b
      
https://github.com/qemu/qemu/commit/c7e7c87ac8e03fe088f4b1f820da30e85a07fd6b
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2-bitmap.c

  Log Message:
  -----------
  qcow2: Switch sectors_covered_by_bitmap_cluster() to byte-based

We are gradually converting to byte-based interfaces, as they are
easier to reason about than sector-based.  Change the qcow2 bitmap
helper function sectors_covered_by_bitmap_cluster(), renaming it
to bytes_covered_by_bitmap_cluster() in the process.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 715a74d819926af38bfeddb3ae29c9fe6b7736bb
      
https://github.com/qemu/qemu/commit/715a74d819926af38bfeddb3ae29c9fe6b7736bb
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/backup.c
    M block/dirty-bitmap.c
    M block/mirror.c
    M block/qcow2-bitmap.c
    M include/block/dirty-bitmap.h

  Log Message:
  -----------
  dirty-bitmap: Set iterator start by offset, not sector

All callers to bdrv_dirty_iter_new() passed 0 for their initial
starting point, drop that parameter.

Most callers to bdrv_set_dirty_iter() were scaling a byte offset to
a sector number; the exception qcow2-bitmap will be converted later
to use byte rather than sector iteration.  Move the scaling to occur
internally to dirty bitmap code instead, so that callers now pass
in bytes.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: f798184cfdcb7f92a38c5f717d675bd75e1fd3ac
      
https://github.com/qemu/qemu/commit/f798184cfdcb7f92a38c5f717d675bd75e1fd3ac
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/backup.c
    M block/dirty-bitmap.c
    M block/mirror.c
    M block/qcow2-bitmap.c

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_dirty_iter_next() to report byte offset

Thanks to recent cleanups, most callers were scaling a return value
of sectors into bytes (the exception, in qcow2-bitmap, will be
converted to byte-based iteration later).  Update the interface to
do the scaling internally instead.

In qcow2-bitmap, the code was specifically checking for an error
return of -1.  To avoid a regression, we either have to make sure
we continue to return -1 (rather than a scaled -512) on error, or
we have to fix the caller to treat all negative values as error
rather than just one magic value.  It's easy enough to make both
changes at the same time, even though either one in isolation
would work.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9a46dba7b76f5198555819905d1d8235947ba98f
      
https://github.com/qemu/qemu/commit/9a46dba7b76f5198555819905d1d8235947ba98f
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/mirror.c
    M migration/block.c

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_get_dirty_count() to report bytes

Thanks to recent cleanups, all callers were scaling a return value
of sectors into bytes; do the scaling internally instead.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3b5d4df0c6b52746c6194bd2ea65828822db8438
      
https://github.com/qemu/qemu/commit/3b5d4df0c6b52746c6194bd2ea65828822db8438
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/mirror.c
    M include/block/dirty-bitmap.h
    M migration/block.c

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_get_dirty_locked() to take bytes

Half the callers were already scaling bytes to sectors; the other
half can eventually be simplified to use byte iteration.  Both
callers were already using the result as a bool, so make that
explicit.  Making the change also makes it easier for a future
dirty-bitmap patch to offload scaling over to the internal hbitmap.

Remember, asking whether a byte is dirty is effectively asking
whether the entire granularity containing the byte is dirty, since
we only track dirtiness by granularity.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Juan Quintela <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: e0d7f73e63427521aabd7311a86fe90fd02897c0
      
https://github.com/qemu/qemu/commit/e0d7f73e63427521aabd7311a86fe90fd02897c0
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/mirror.c
    M include/block/dirty-bitmap.h
    M migration/block.c

  Log Message:
  -----------
  dirty-bitmap: Change bdrv_[re]set_dirty_bitmap() to use bytes

Some of the callers were already scaling bytes to sectors; others
can be easily converted to pass byte offsets, all in our shift
towards a consistent byte interface everywhere.  Making the change
will also make it easier to write the hold-out callers to use byte
rather than sectors for their iterations; it also makes it easier
for a future dirty-bitmap patch to offload scaling over to the
internal hbitmap.  Although all callers happen to pass
sector-aligned values, make the internal scaling robust to any
sub-sector requests.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 23ca459a455c83ffadb03ab1eedf0b6cff62bfeb
      
https://github.com/qemu/qemu/commit/23ca459a455c83ffadb03ab1eedf0b6cff62bfeb
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  mirror: Switch mirror_dirty_init() to byte-based iteration

Now that we have adjusted the majority of the calls this function
makes to be byte-based, it is easier to read the code if it makes
passes over the image using bytes rather than sectors.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: b85ee45334585c49c5489a303336df119408763a
      
https://github.com/qemu/qemu/commit/b85ee45334585c49c5489a303336df119408763a
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: Switch qcow2_measure() to byte-based iteration

This is new code, but it is easier to read if it makes passes over
the image using bytes rather than sectors (and will get easier in
the future when bdrv_get_block_status is converted to byte-based).

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ab94db6f7609e562b5f68cc71807d5d50e24e224
      
https://github.com/qemu/qemu/commit/ab94db6f7609e562b5f68cc71807d5d50e24e224
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2-bitmap.c

  Log Message:
  -----------
  qcow2: Switch load_bitmap_data() to byte-based iteration

Now that we have adjusted the majority of the calls this function
makes to be byte-based, it is easier to read the code if it makes
passes over the image using bytes rather than sectors.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 49d741b5041b79214db58f364cebe2f367517711
      
https://github.com/qemu/qemu/commit/49d741b5041b79214db58f364cebe2f367517711
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2-bitmap.c
    M tests/qemu-iotests/165

  Log Message:
  -----------
  qcow2: Switch store_bitmap_data() to byte-based iteration

Now that we have adjusted the majority of the calls this function
makes to be byte-based, it is easier to read the code if it makes
passes over the image using bytes rather than sectors.

iotests 165 was rather weak - on a default 64k-cluster image, where
bitmap granularity also defaults to 64k bytes, a single cluster of
the bitmap table thus covers (64*1024*8) bits which each cover 64k
bytes, or 32G of image space.  But the test only uses a 1G image,
so it cannot trigger any more than one loop of the code in
store_bitmap_data(); and it was writing to the first cluster.  In
order to test that we are properly aligning which portions of the
bitmap are being written to the file, we really want to test a case
where the first dirty bit returned by bdrv_dirty_iter_next() is not
aligned to the start of a cluster, which we can do by modifying the
test to write data that doesn't happen to fall in the first cluster
of the image.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0fdf1a4f68e5f27df9bb10a7223bd519c8d30e31
      
https://github.com/qemu/qemu/commit/0fdf1a4f68e5f27df9bb10a7223bd519c8d30e31
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c
    M block/io.c
    M include/block/block_int.h

  Log Message:
  -----------
  dirty-bitmap: Switch bdrv_set_dirty() to bytes

Both callers already had bytes available, but were scaling to
sectors.  Move the scaling to internal code.  In the case of
bdrv_aligned_pwritev(), we are now passing the exact offset
rather than a rounded sector-aligned value, but that's okay
as long as dirty bitmap widens start/bytes to granularity
boundaries.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ca759622441da1d19107efaa62d9d63fb83ebb79
      
https://github.com/qemu/qemu/commit/ca759622441da1d19107efaa62d9d63fb83ebb79
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/dirty-bitmap.c

  Log Message:
  -----------
  dirty-bitmap: Convert internal hbitmap size/granularity

Now that all callers are using byte-based interfaces, there's no
reason for our internal hbitmap to remain with sector-based
granularity.  It also simplifies our internal scaling, since we
already know that hbitmap widens requests out to granularity
boundaries.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Fam Zheng <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: dbfa934106d22402d809d039e773b50ab1885477
      
https://github.com/qemu/qemu/commit/dbfa934106d22402d809d039e773b50ab1885477
  Author: Thomas Huth <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M hw/block/onenand.c

  Log Message:
  -----------
  hw/block/onenand: Remove dead code block

The condition of the for-loop makes sure that b is always smaller
than s->blocks, so the "if (b >= s->blocks)" statement is completely
superfluous here.

Buglink: https://bugs.launchpad.net/qemu/+bug/1715007
Signed-off-by: Thomas Huth <address@hidden>
Reviewed-by: Laurent Vivier <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: f06a8dcfc6d51cdcd51f4320b895680cbbc872a9
      
https://github.com/qemu/qemu/commit/f06a8dcfc6d51cdcd51f4320b895680cbbc872a9
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common
    M tests/qemu-iotests/common.config
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: remove dead code

This includes shell function, shell variables and command line options
(randomize.awk does not exist).

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9ee4b6f803335d23e3b6fe21c139707918c3eb24
      
https://github.com/qemu/qemu/commit/9ee4b6f803335d23e3b6fe21c139707918c3eb24
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common
    M tests/qemu-iotests/common.config

  Log Message:
  -----------
  qemu-iotests: get rid of AWK_PROG

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 48259488aa0aa05047a27f6b53a8df0111e32472
      
https://github.com/qemu/qemu/commit/48259488aa0aa05047a27f6b53a8df0111e32472
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/common
    M tests/qemu-iotests/common.config
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: move "check" code out of common.rc

Some functions in common.rc are never used by the tests.  Move
them out of that file and into common, which is already included
only by "check".

Code that actually *is* common to "check" and tests can be placed in
common.config.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: cceaf1db6fbede3ed0bca103d12f8a19c1374e42
      
https://github.com/qemu/qemu/commit/cceaf1db6fbede3ed0bca103d12f8a19c1374e42
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common
    M tests/qemu-iotests/common.config

  Log Message:
  -----------
  qemu-iotests: cleanup and fix search for programs

Instead of ./check failing when a binary is missing, we try each test
case now and each one fails with tons of test case diffs.  Also, all the
variables were initialized by "check" prior to "common" being sourced,
and then (uselessly) checked for emptiness again in "check".

Centralize the search for programs in "common" (which will soon be
one with "check"), including the "realpath" invocation which can be done
just once in "check" rather than in the tests.

For qnio_server, move the detection to "common", simplifying
set_prog_path to stop handling the unused second argument, and
embedding the "realpath" pass.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d1f2447a3e8d0804838e2445f955996c925395ef
      
https://github.com/qemu/qemu/commit/d1f2447a3e8d0804838e2445f955996c925395ef
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/039.out
    M tests/qemu-iotests/061.out
    M tests/qemu-iotests/137.out
    M tests/qemu-iotests/common.config
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: limit non-_PROG-suffixed variables to common.rc

These are never used by "check", with one exception that does not need
$QEMU_OPTIONS.  Keep them in common.rc, which will be soon included only
by the tests.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 3817ce03bf3373065c25e389ecb7f23a511ebcdc
      
https://github.com/qemu/qemu/commit/3817ce03bf3373065c25e389ecb7f23a511ebcdc
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: do not include common.rc in "check"

It only provides functions used by the test programs.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: cce293a29459e1fd5f74e55d87dc9c25991da578
      
https://github.com/qemu/qemu/commit/cce293a29459e1fd5f74e55d87dc9c25991da578
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/common
    M tests/qemu-iotests/common.config
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: disintegrate more parts of common.config

Split "check" parts from tests part.

For the directory setup, the actual computation of directories goes
in "check", while the sanity checks go in the tests.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 8f4dcaba9bed36d067f2f2da1e2ceb9d1b50f2b2
      
https://github.com/qemu/qemu/commit/8f4dcaba9bed36d067f2f2da1e2ceb9d1b50f2b2
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common

  Log Message:
  -----------
  qemu-iotests: fix uninitialized variable

The variable is used in "common" but defined only after the file
is sourced.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 4e670492efe29b809f5b1d88eee2adcdf70d8d13
      
https://github.com/qemu/qemu/commit/4e670492efe29b809f5b1d88eee2adcdf70d8d13
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    M tests/qemu-iotests/common.rc

  Log Message:
  -----------
  qemu-iotests: get rid of $iam

The variable is almost unused, and one of the two uses is actually
uninitialized.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 09d653e6176a5e92f5d2a3b2270f386d9ce6a544
      
https://github.com/qemu/qemu/commit/09d653e6176a5e92f5d2a3b2270f386d9ce6a544
  Author: Paolo Bonzini <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check
    R tests/qemu-iotests/common

  Log Message:
  -----------
  qemu-iotests: merge "check" and "common"

"check" is full of qemu-iotests--specific details.  Separating it
from "common" does not make much sense anymore.

Signed-off-by: Paolo Bonzini <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 6858eba09ed69e64c8d05d4f4b8167b42a011b7f
      
https://github.com/qemu/qemu/commit/6858eba09ed69e64c8d05d4f4b8167b42a011b7f
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c
    M include/block/block_int.h

  Log Message:
  -----------
  block: Introduce BdrvChildRole.update_filename

There is no good reason for bdrv_drop_intermediate() to know the active
layer above the subchain it is operating on - even more so, because
the assumption that there is a single active layer above it is not
generally true.

In order to prepare removal of the active parameter, use a BdrvChildRole
callback to update the backing file string in the overlay image instead
of directly calling bdrv_change_backing_file().

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 61f09cea01391eaa23ea3bc78ab37a7d2da565fb
      
https://github.com/qemu/qemu/commit/61f09cea01391eaa23ea3bc78ab37a7d2da565fb
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c
    M block/commit.c

  Log Message:
  -----------
  commit: Support multiple roots above top node

This changes the commit block job to support operation in a graph where
there is more than a single active layer that references the top node.

This involves inserting the commit filter node not only on the path
between the given active node and the top node, but between the top node
and all of its parents.

On completion, bdrv_drop_intermediate() must consider all parents for
updating the backing file link. These parents may be backing files
themselves and as such read-only; reopen them temporarily if necessary.
Previously this was achieved by the bdrv_reopen() calls in the commit
block job that made overlay_bs read-write for the whole duration of the
block job, even though write access is only needed on completion.

Now that we consider all parents, overlay_bs is meaningless. It is left
in place in this commit, but we'll remove it soon.

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 72538537d818d2870104448ba819b91dc70c241a
      
https://github.com/qemu/qemu/commit/72538537d818d2870104448ba819b91dc70c241a
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/common.qemu

  Log Message:
  -----------
  qemu-iotests: Allow QMP pretty printing in common.qemu

QMP responses to certain commands can become quite long, which doesn't
only make reading them hard, but also means that the maximum line length
in patch emails can be exceeded. Allow tests to switch to QMP pretty
printing, which results in more, but shorter lines.

We also need to make sure to keep indentation in the response for this
to work as expected.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 7c61a4a3f90763527f212abc6d28e36df9e32202
      
https://github.com/qemu/qemu/commit/7c61a4a3f90763527f212abc6d28e36df9e32202
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    A tests/qemu-iotests/191
    A tests/qemu-iotests/191.out
    M tests/qemu-iotests/group

  Log Message:
  -----------
  qemu-iotests: Test commit block job where top has two parents

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: bde70715b67cc5183b00b445b811c1dfc0f74d2e
      
https://github.com/qemu/qemu/commit/bde70715b67cc5183b00b445b811c1dfc0f74d2e
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c
    M block/commit.c
    M include/block/block.h
    M tests/qemu-iotests/030

  Log Message:
  -----------
  commit: Remove overlay_bs

We don't need to make any assumptions about the graph layout above the
top node of the commit operation any more. Remove the use of
bdrv_find_overlay() and related variables from the commit job code.

bdrv_drop_intermediate() doesn't use the 'active' parameter any more, so
we can just drop it.

The overlay node was previously added to the block job to get a
BLK_PERM_GRAPH_MOD. We really need to respect those permissions in
bdrv_drop_intermediate() now, but as long as we haven't figured out yet
how BLK_PERM_GRAPH_MOD is actually supposed to work, just leave a TODO
comment there.

With this change, it is now possible to perform another block job on an
overlay node without conflicts. qemu-iotests 030 is changed accordingly.

Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Eric Blake <address@hidden>


  Commit: 0f40444cc4fc6526a9a544b11475f1086113f9ba
      
https://github.com/qemu/qemu/commit/0f40444cc4fc6526a9a544b11475f1086113f9ba
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M qemu-io.c

  Log Message:
  -----------
  qemu-io: Add -C for opening with copy-on-read

Make it easier to enable copy-on-read during iotests, by
exposing a new bool option to main and open.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 9cdcfd9f7afd0274919af95a164178ac6ee847ca
      
https://github.com/qemu/qemu/commit/9cdcfd9f7afd0274919af95a164178ac6ee847ca
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Uniform handling of 0-length bdrv_get_block_status()

Handle a 0-length block status request up front, with a uniform
return value claiming the area is not allocated.

Most callers don't pass a length of 0 to bdrv_get_block_status()
and friends; but it definitely happens with a 0-length read when
copy-on-read is enabled.  While we could audit all callers to
ensure that they never make a 0-length request, and then assert
that fact, it was just as easy to fix things to always report
success (as long as the callers are careful to not go into an
infinite loop).  However, we had inconsistent behavior on whether
the status is reported as allocated or defers to the backing
layer, depending on what callbacks the driver implements, and
possibly wasting quite a few CPU cycles to get to that answer.
Consistently reporting unallocated up front doesn't really hurt
anything, and makes it easier both for callers (0-length requests
now have well-defined behavior) and for drivers (drivers don't
have to deal with 0-length requests).

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 8803714b53243816e96a759de3cfd22625230023
      
https://github.com/qemu/qemu/commit/8803714b53243816e96a759de3cfd22625230023
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/check

  Log Message:
  -----------
  iotests: Restore stty settings on completion

Executing qemu with a terminal as stdin will temporarily alter stty
settings on that terminal (for example, disabling echo), because of
how we run both the monitor and any multiplexing with guest input.
Normally, qemu restores the original settings on exit; but if an
iotest triggers qemu to abort in the middle, we can be left with
the altered terminal setup.  This can make life very annoying when
debugging an iotest failure (not everyone remembers the trick of
blind-typing 'stty sane' without echo, and some people prefer
terminal settings that are slightly different than the defaults
picked by 'stty sane').

It is possible to avoid qemu corrupting the terminal by not passing
a terminal to qemu's stdin in the first place (as in, use
'./check ... </dev/null'), but that's extra typing to have to
remember.  But running 'exec </dev/null' in the harness seems like
it might be too heavy of a hammer.  So I instead went the the
solution of saving and restoring the stty settings, only when the
harness detects that it is run interactively.

I tested this patch by forcing an allocation failure (I can't
guarantee that this particular limit will work on all setups, but
it shows the idea):
 $ (ulimit -S -v 500000; ./check -qcow2 1)

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d855ebcd3cca4080a81aeec9c0a27af006734280
      
https://github.com/qemu/qemu/commit/d855ebcd3cca4080a81aeec9c0a27af006734280
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/io.c
    M qapi/block-core.json

  Log Message:
  -----------
  block: Add blkdebug hook for copy-on-read

Make it possible to inject errors on writes performed during a
read operation due to copy-on-read semantics.

Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Jeff Cody <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: cb2e28780c7080af489e72227683fe374f05022d
      
https://github.com/qemu/qemu/commit/cb2e28780c7080af489e72227683fe374f05022d
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/io.c

  Log Message:
  -----------
  block: Perform copy-on-read in loop

Improve our braindead copy-on-read implementation.  Pre-patch,
we have multiple issues:
- we create a bounce buffer and perform a write for the entire
request, even if the active image already has 99% of the
clusters occupied, and really only needs to copy-on-read the
remaining 1% of the clusters
- our bounce buffer was as large as the read request, and can
needlessly exhaust our memory by using double the memory of
the request size (the original request plus our bounce buffer),
rather than a capped maximum overhead beyond the original
- if a driver has a max_transfer limit, we are bypassing the
normal code in bdrv_aligned_preadv() that fragments to that
limit, and instead attempt to read the entire buffer from the
driver in one go, which some drivers may assert on
- a client can request a large request of nearly 2G such that
rounding the request out to cluster boundaries results in a
byte count larger than 2G.  While this cannot exceed 32 bits,
it DOES have some follow-on problems:
-- the call to bdrv_driver_pread() can assert for exceeding
BDRV_REQUEST_MAX_BYTES, if the driver is old and lacks
.bdrv_co_preadv
-- if the buffer is all zeroes, the subsequent call to
bdrv_co_do_pwrite_zeroes is a no-op due to a negative size,
which means we did not actually copy on read

Fix all of these issues by breaking up the action into a loop,
where each iteration is capped to sane limits.  Also, querying
the allocation status allows us to optimize: when data is
already present in the active layer, we don't need to bounce.

Note that the code has a telling comment that copy-on-read
should probably be a filter driver rather than a bolt-on hack
in io.c; but that remains a task for another day.

CC: address@hidden
Signed-off-by: Eric Blake <address@hidden>
Reviewed-by: Kevin Wolf <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 461743390d3a1ceafa4503811adbc87c7d372741
      
https://github.com/qemu/qemu/commit/461743390d3a1ceafa4503811adbc87c7d372741
  Author: Eric Blake <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    A tests/qemu-iotests/197
    A tests/qemu-iotests/197.out
    M tests/qemu-iotests/common.filter
    M tests/qemu-iotests/group

  Log Message:
  -----------
  iotests: Add test 197 for covering copy-on-read

Add a test for qcow2 copy-on-read behavior, including exposure
for the just-fixed bugs.

The copy-on-read behavior is always to a qcow2 image, but the
test is careful to allow running with most image protocol/format
combos as the backing file being copied from (luks being the
exception, as it is harder to pass the right secret to all the
right places).  In fact, for './check nbd', this appears to be
the first time we've had a qcow2 image wrapping NBD, requiring
an additional line in _filter_img_create to match the similar
line in _filter_img_info.

Invoking blkdebug to prove we don't write too much took some
effort to get working; and it requires that $TEST_WRAP (based
on $TEST_DIR) not be subject to word splitting.  We may decide
later to have the entire iotests suite use relative rather than
absolute names, to avoid problems inherited by the absolute
name of $PWD or $TEST_DIR, at which point the sanity check in
this commit could be simplified.

This test requires at least 2G of consecutive memory to succeed;
as such, it is prone to spurious failures, particularly on
32-bit machines under load.  This situation is detected and
triggers an early exit to skip the test, rather than a failure.
To manually provoke this setup on a beefier machine, I used:
  $ (ulimit -S -v 1000000; ./check -qcow2 197)

Signed-off-by: Eric Blake <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 161253e2d0a83a1b33bca019c6e926013e1a03db
      
https://github.com/qemu/qemu/commit/161253e2d0a83a1b33bca019c6e926013e1a03db
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  block: use 1 MB bounce buffers for crypto instead of 16KB

Using 16KB bounce buffers creates a significant performance
penalty for I/O to encrypted volumes on storage which high
I/O latency (rotating rust & network drives), because it
triggers lots of fairly small I/O operations.

On tests with rotating rust, and cache=none|directsync,
write speed increased from 2MiB/s to 32MiB/s, on a par
with that achieved by the in-kernel luks driver. With
other cache modes the in-kernel driver is still notably
faster because it is able to report completion of the
I/O request before any encryption is done, while the
in-QEMU driver must encrypt the data before completion.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 850f49de9b57511dcaf2cd7e45059f8f38fadf3b
      
https://github.com/qemu/qemu/commit/850f49de9b57511dcaf2cd7e45059f8f38fadf3b
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M crypto/block-luks.c
    M crypto/block-qcow.c
    M crypto/block.c
    M crypto/blockpriv.h
    M include/crypto/block.h

  Log Message:
  -----------
  crypto: expose encryption sector size in APIs

While current encryption schemes all have a fixed sector size of
512 bytes, this is not guaranteed to be the case in future. Expose
the sector size in the APIs so the block layer can remove assumptions
about fixed 512 byte sectors.

Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 31376555c7b447afb1bf9084eacbb8f566ff6b9d
      
https://github.com/qemu/qemu/commit/31376555c7b447afb1bf9084eacbb8f566ff6b9d
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  block: fix data type casting for crypto payload offset

The crypto APIs report the offset of the data payload as an uint64_t
type, but the block driver is casting to size_t or ssize_t which will
potentially truncate.

Most of the block APIs use int64_t for offsets meanwhile, so even if
using uint64_t in the crypto block driver we are still at risk of
truncation.

Change the block crypto driver to use uint64_t, but add asserts that
the value is less than INT64_MAX.

Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: a73466fbad6d48ba356940474cd72da602373304
      
https://github.com/qemu/qemu/commit/a73466fbad6d48ba356940474cd72da602373304
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  block: convert crypto driver to bdrv_co_preadv|pwritev

Make the crypto driver implement the bdrv_co_preadv|pwritev
callbacks, and also use bdrv_co_preadv|pwritev for I/O
with the protocol driver beneath. This replaces sector based
I/O with byte based I/O, and allows us to stop assuming the
physical sector size matches the encryption sector size.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 4609742a495d98ac358098e10d91890185dcdc60
      
https://github.com/qemu/qemu/commit/4609742a495d98ac358098e10d91890185dcdc60
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c
    M block/qcow.c
    M block/qcow2-cluster.c
    M block/qcow2.c
    M crypto/block-luks.c
    M crypto/block-qcow.c
    M crypto/block.c
    M crypto/blockpriv.h
    M include/crypto/block.h

  Log Message:
  -----------
  block: convert qcrypto_block_encrypt|decrypt to take bytes offset

Instead of sector offset, take the bytes offset when encrypting
or decrypting data.

Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: d67a6b09b4ac27a4fac07544ded79b40d2717a0d
      
https://github.com/qemu/qemu/commit/d67a6b09b4ac27a4fac07544ded79b40d2717a0d
  Author: Daniel P. Berrange <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  block: support passthrough of BDRV_REQ_FUA in crypto driver

The BDRV_REQ_FUA flag can trivially be allowed in the crypt driver
as a passthrough to the underlying block driver.

Reviewed-by: Max Reitz <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Daniel P. Berrange <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 18775ff32697ab6e1fd47989673bf1de54d0d942
      
https://github.com/qemu/qemu/commit/18775ff32697ab6e1fd47989673bf1de54d0d942
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  block/mirror: check backing in bdrv_mirror_top_refresh_filename

Backing may be zero after failed bdrv_attach_child in
bdrv_set_backing_hd, which leads to SIGSEGV.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Reviewed-by: John Snow <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 47500c6775813c8f2b5a5de04d84222f3cecc62d
      
https://github.com/qemu/qemu/commit/47500c6775813c8f2b5a5de04d84222f3cecc62d
  Author: Max Reitz <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M tests/qemu-iotests/195

  Log Message:
  -----------
  iotests: Fix 195 if IMGFMT is part of TEST_DIR

do_run_qemu() in iotest 195 first applies _filter_imgfmt when printing
qemu's command line and _filter_testdir only afterwards.  Therefore, if
the image format is part of the test directory path, _filter_testdir
will no longer apply and the actual output will differ from the
reference output even in case of success.

For example, TEST_DIR might be "/tmp/test-qcow2", in which case
_filter_imgfmt first transforms this to "/tmp/test-IMGFMT" which is no
longer recognized as the TEST_DIR by _filter_testdir.

Fix this by not applying _filter_imgfmt in do_run_qemu() but in
run_qemu() instead, and only after _filter_testdir.

Signed-off-by: Max Reitz <address@hidden>
Message-id: address@hidden
Reviewed-by: Eric Blake <address@hidden>
Signed-off-by: Max Reitz <address@hidden>


  Commit: 76a2a30a99c670e9ec1b4a5d976868059c6bc258
      
https://github.com/qemu/qemu/commit/76a2a30a99c670e9ec1b4a5d976868059c6bc258
  Author: Pavel Butsykin <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: fix return error code in qcow2_truncate()

Signed-off-by: Pavel Butsykin <address@hidden>
Reviewed-by: Eric Blake <address@hidden>
Reviewed-by: John Snow <address@hidden>
Reviewed-by: Max Reitz <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: 163bc39d2c2921430e5c23f4d0a0966d62f67a02
      
https://github.com/qemu/qemu/commit/163bc39d2c2921430e5c23f4d0a0966d62f67a02
  Author: Pavel Butsykin <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: truncate the tail of the image file after shrinking the image

Now after shrinking the image, at the end of the image file, there might be a
tail that probably will never be used. So we can find the last used cluster and
cut the tail.

Signed-off-by: Pavel Butsykin <address@hidden>
Reviewed-by: John Snow <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: ce960aa9062a407d0ca15aee3dcd3bd84a4e24f9
      
https://github.com/qemu/qemu/commit/ce960aa9062a407d0ca15aee3dcd3bd84a4e24f9
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/mirror.c

  Log Message:
  -----------
  block/mirror: check backing in bdrv_mirror_top_flush

Backing may be zero after failed bdrv_append in mirror_start_job,
which leads to SIGSEGV.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Message-id: address@hidden
Signed-off-by: Max Reitz <address@hidden>


  Commit: fc3fd63fc0573ffd2ee569591a2e7f6c7310fd18
      
https://github.com/qemu/qemu/commit/fc3fd63fc0573ffd2ee569591a2e7f6c7310fd18
  Author: Kevin Wolf <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block/crypto.c
    M block/mirror.c
    M block/qcow.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h
    M crypto/block-luks.c
    M crypto/block-qcow.c
    M crypto/block.c
    M crypto/blockpriv.h
    M include/crypto/block.h
    M tests/qemu-iotests/195

  Log Message:
  -----------
  Merge remote-tracking branch 'mreitz/tags/pull-block-2017-10-06' into 
queue-block

Block patches

# gpg: Signature made Fri Oct  6 16:30:57 2017 CEST
# gpg:                using RSA key F407DB0061D5CF40
# gpg: Good signature from "Max Reitz <address@hidden>"
# Primary key fingerprint: 91BE B60A 30DB 3E88 57D1  1829 F407 DB00 61D5 CF40

* mreitz/tags/pull-block-2017-10-06:
  block/mirror: check backing in bdrv_mirror_top_flush
  qcow2: truncate the tail of the image file after shrinking the image
  qcow2: fix return error code in qcow2_truncate()
  iotests: Fix 195 if IMGFMT is part of TEST_DIR
  block/mirror: check backing in bdrv_mirror_top_refresh_filename
  block: support passthrough of BDRV_REQ_FUA in crypto driver
  block: convert qcrypto_block_encrypt|decrypt to take bytes offset
  block: convert crypto driver to bdrv_co_preadv|pwritev
  block: fix data type casting for crypto payload offset
  crypto: expose encryption sector size in APIs
  block: use 1 MB bounce buffers for crypto instead of 16KB

Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 530049bc1dcc24c1178a29d99ca08b6dd08413e0
      
https://github.com/qemu/qemu/commit/530049bc1dcc24c1178a29d99ca08b6dd08413e0
  Author: Peter Maydell <address@hidden>
  Date:   2017-10-06 (Fri, 06 Oct 2017)

  Changed paths:
    M block.c
    M block/backup.c
    M block/commit.c
    M block/crypto.c
    M block/dirty-bitmap.c
    M block/io.c
    M block/mirror.c
    M block/qcow.c
    M block/qcow2-bitmap.c
    M block/qcow2-cluster.c
    M block/qcow2-refcount.c
    M block/qcow2.c
    M block/qcow2.h
    M crypto/block-luks.c
    M crypto/block-qcow.c
    M crypto/block.c
    M crypto/blockpriv.h
    M hw/block/onenand.c
    M include/block/block.h
    M include/block/block_int.h
    M include/block/dirty-bitmap.h
    M include/crypto/block.h
    M include/qemu/hbitmap.h
    M migration/block.c
    M qapi/block-core.json
    M qemu-io.c
    M tests/qemu-iotests/030
    M tests/qemu-iotests/039.out
    M tests/qemu-iotests/061.out
    M tests/qemu-iotests/137.out
    M tests/qemu-iotests/165
    A tests/qemu-iotests/191
    A tests/qemu-iotests/191.out
    M tests/qemu-iotests/195
    A tests/qemu-iotests/197
    A tests/qemu-iotests/197.out
    M tests/qemu-iotests/check
    R tests/qemu-iotests/common
    M tests/qemu-iotests/common.config
    M tests/qemu-iotests/common.filter
    M tests/qemu-iotests/common.qemu
    M tests/qemu-iotests/common.rc
    M tests/qemu-iotests/group
    M tests/test-hbitmap.c
    M util/hbitmap.c

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches

# gpg: Signature made Fri 06 Oct 2017 16:52:59 BST
# gpg:                using RSA key 0x7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream: (54 commits)
  block/mirror: check backing in bdrv_mirror_top_flush
  qcow2: truncate the tail of the image file after shrinking the image
  qcow2: fix return error code in qcow2_truncate()
  iotests: Fix 195 if IMGFMT is part of TEST_DIR
  block/mirror: check backing in bdrv_mirror_top_refresh_filename
  block: support passthrough of BDRV_REQ_FUA in crypto driver
  block: convert qcrypto_block_encrypt|decrypt to take bytes offset
  block: convert crypto driver to bdrv_co_preadv|pwritev
  block: fix data type casting for crypto payload offset
  crypto: expose encryption sector size in APIs
  block: use 1 MB bounce buffers for crypto instead of 16KB
  iotests: Add test 197 for covering copy-on-read
  block: Perform copy-on-read in loop
  block: Add blkdebug hook for copy-on-read
  iotests: Restore stty settings on completion
  block: Uniform handling of 0-length bdrv_get_block_status()
  qemu-io: Add -C for opening with copy-on-read
  commit: Remove overlay_bs
  qemu-iotests: Test commit block job where top has two parents
  qemu-iotests: Allow QMP pretty printing in common.qemu
  ...

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/5121d81e387b...530049bc1dcc