[Qemu-commits] [qemu/qemu] 884813: qemu-img: allow compressed not-in-order writes

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: 88481329c0a43373f994f0c8ed19e888a8c86830
      
https://github.com/qemu/qemu/commit/88481329c0a43373f994f0c8ed19e888a8c86830
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M qemu-img.c

  Log Message:
  -----------
  qemu-img: allow compressed not-in-order writes

No reason to forbid them, and they are needed to improve performance
with compress-threads in further patches.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 2714f13d69adf73638842729ccfb3bdd6d5ee98f
      
https://github.com/qemu/qemu/commit/2714f13d69adf73638842729ccfb3bdd6d5ee98f
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/qcow2.c

  Log Message:
  -----------
  qcow2: refactor data compression

Make a separate function for compression to be parallelized later.
 - use .avail_out field instead of .next_out to calculate size of
   compressed data. It looks more natural and it allows to keep dest to
   be void pointer
 - set avail_out to be at least one byte less than input, to be sure
   avoid inefficient compression earlier

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: ceb029cd6feccf9f7607833b71dd609d149421a1
      
https://github.com/qemu/qemu/commit/ceb029cd6feccf9f7607833b71dd609d149421a1
  Author: Vladimir Sementsov-Ogievskiy <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/qcow2.c
    M block/qcow2.h

  Log Message:
  -----------
  qcow2: add compress threads

Do data compression in separate threads. This significantly improve
performance for qemu-img convert with -W (allow async writes) and -c
(compressed) options.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 7ae9f3f61b2b99e2f348d3dc4a4ef2c6af0ae9bc
      
https://github.com/qemu/qemu/commit/7ae9f3f61b2b99e2f348d3dc4a4ef2c6af0ae9bc
  Author: Ari Sundholm <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block.c
    M include/block/block.h

  Log Message:
  -----------
  block: Move two block permission constants to the relevant enum

This allows using the two constants outside of block.c, which will
happen in a subsequent patch.

Signed-off-by: Ari Sundholm <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: bfcc224e3cf04ee3fef0eb69984607b5764d9892
      
https://github.com/qemu/qemu/commit/bfcc224e3cf04ee3fef0eb69984607b5764d9892
  Author: Aapo Vienamo <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M MAINTAINERS
    M block/Makefile.objs
    A block/blklogwrites.c
    M qapi/block-core.json

  Log Message:
  -----------
  block: Add blklogwrites

Implements a block device write logging system, similar to Linux kernel
device mapper dm-log-writes. The write operations that are performed
on a block device are logged to a file or another block device. The
write log format is identical to the dm-log-writes format. Currently,
log markers are not supported.

This functionality can be used for crash consistency and fs consistency
testing. By implementing it in qemu, tests utilizing write logs can be
be used to test non-Linux drivers and older kernels.

The driver accepts an optional parameter to set the sector size used
for logging. This makes the driver require all requests to be aligned
to this sector size and also makes offsets and sizes of writes in the
log metadata to be expressed in terms of this value (the log format has
a granularity of one sector for offsets and sizes). This allows
accurate logging of writes to guest block devices that have unusual
sector sizes.

The implementation is based on the blkverify and blkdebug block
drivers.

Signed-off-by: Aapo Vienamo <address@hidden>
Signed-off-by: Ari Sundholm <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 824808dd77821ceba05357cb1ee4069a6a95bebd
      
https://github.com/qemu/qemu/commit/824808dd77821ceba05357cb1ee4069a6a95bebd
  Author: Kevin Wolf <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block.c
    M tests/qemu-iotests/051
    M tests/qemu-iotests/051.out
    M tests/qemu-iotests/051.pc.out

  Log Message:
  -----------
  block: Don't silently truncate node names

If the user passes a too long node name string, we silently truncate it
to fit into BlockDriverState.node_name, i.e. to 31 characters. Apart
from surprising the user when the node has a different name than
requested, this also bypasses the check for duplicate names, so that the
same name can be assigned to multiple nodes.

Fix this by just making too long node names an error.

Reported-by: Peter Krempa <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0b68589d17ef2ea78b3cf016525db13cee1d99e7
      
https://github.com/qemu/qemu/commit/0b68589d17ef2ea78b3cf016525db13cee1d99e7
  Author: Kevin Wolf <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/crypto.c

  Log Message:
  -----------
  block/crypto: Fix memory leak in create error path

Fixes: Coverity CID 1393782
Signed-off-by: Kevin Wolf <address@hidden>
Reviewed-by: Daniel P. Berrangé <address@hidden>


  Commit: 2dacaf7c82c2771d507e5e59efcea78d933baca9
      
https://github.com/qemu/qemu/commit/2dacaf7c82c2771d507e5e59efcea78d933baca9
  Author: Ari Sundholm <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/blklogwrites.c

  Log Message:
  -----------
  block/blklogwrites: Change log_sector_size from int64_t to uint64_t

This was a simple oversight when working on intermediate versions
of the original patch which introduced blklogwrites.

Signed-off-by: Ari Sundholm <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 0878b3c113145d4e01d65aadd4efaf097a3fda4b
      
https://github.com/qemu/qemu/commit/0878b3c113145d4e01d65aadd4efaf097a3fda4b
  Author: Ari Sundholm <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/blklogwrites.c
    M qapi/block-core.json

  Log Message:
  -----------
  block/blklogwrites: Add an option for appending to an old log

Suggested by Kevin Wolf. May be useful when testing multiple batches
of writes or doing long-term testing involving restarts of the VM.

Signed-off-by: Ari Sundholm <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1dce698ea85bb18f62e4c540d4db628bacfba6ba
      
https://github.com/qemu/qemu/commit/1dce698ea85bb18f62e4c540d4db628bacfba6ba
  Author: Ari Sundholm <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/blklogwrites.c
    M qapi/block-core.json

  Log Message:
  -----------
  block/blklogwrites: Add an option for the update interval of the log 
superblock

This is a way to ensure that the log superblock is periodically
updated. Before, this was only done on flush requests, which may
not be enough if the VM exits abnormally, omitting the final flush.

The default interval is 4096 write requests.

Signed-off-by: Ari Sundholm <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: d815efcaf01b1698e2fdf0f3e125201025c53191
      
https://github.com/qemu/qemu/commit/d815efcaf01b1698e2fdf0f3e125201025c53191
  Author: Max Reitz <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  file-posix: Fix creation locking

raw_apply_lock_bytes() takes a bit mask of "permissions that are NOT
shared".

Also, make the "perm" and "shared" variables uint64_t, because I do not
particularly like using ~ on signed integers (and other permission masks
are usually uint64_t, too).

Reported-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 7c20c808a5cbf5d244735bc78fc3138c739c1946
      
https://github.com/qemu/qemu/commit/7c20c808a5cbf5d244735bc78fc3138c739c1946
  Author: Max Reitz <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M block/file-posix.c

  Log Message:
  -----------
  file-posix: Unlock FD after creation

Closing the FD does not necessarily mean that it is unlocked.  Fix this
by relinquishing all permission locks before qemu_close().

Reported-by: Kevin Wolf <address@hidden>
Signed-off-by: Max Reitz <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>


  Commit: 1daf14ec9e66cbe3cbfa74ba17c6cdd9cc6d8e07
      
https://github.com/qemu/qemu/commit/1daf14ec9e66cbe3cbfa74ba17c6cdd9cc6d8e07
  Author: Peter Maydell <address@hidden>
  Date:   2018-07-05 (Thu, 05 Jul 2018)

  Changed paths:
    M MAINTAINERS
    M block.c
    M block/Makefile.objs
    A block/blklogwrites.c
    M block/crypto.c
    M block/file-posix.c
    M block/qcow2.c
    M block/qcow2.h
    M include/block/block.h
    M qapi/block-core.json
    M qemu-img.c
    M tests/qemu-iotests/051
    M tests/qemu-iotests/051.out
    M tests/qemu-iotests/051.pc.out

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging

Block layer patches:

- qcow2: Use worker threads for compression to improve performance of
  'qemu-img convert -W' and compressed backup jobs
- blklogwrites: New filter driver to log write requests to an image in
  the dm-log-writes format
- file-posix: Fix image locking during image creation
- crypto: Fix memory leak in error path
- Error out instead of silently truncating node names

# gpg: Signature made Thu 05 Jul 2018 11:24:33 BST
# gpg:                using RSA key 7F09B272C88F2FD6
# gpg: Good signature from "Kevin Wolf <address@hidden>"
# Primary key fingerprint: DC3D EB15 9A9A F95D 3D74  56FE 7F09 B272 C88F 2FD6

* remotes/kevin/tags/for-upstream:
  file-posix: Unlock FD after creation
  file-posix: Fix creation locking
  block/blklogwrites: Add an option for the update interval of the log 
superblock
  block/blklogwrites: Add an option for appending to an old log
  block/blklogwrites: Change log_sector_size from int64_t to uint64_t
  block/crypto: Fix memory leak in create error path
  block: Don't silently truncate node names
  block: Add blklogwrites
  block: Move two block permission constants to the relevant enum
  qcow2: add compress threads
  qcow2: refactor data compression
  qemu-img: allow compressed not-in-order writes

Signed-off-by: Peter Maydell <address@hidden>


Compare: https://github.com/qemu/qemu/compare/6cf495be0b44...1daf14ec9e66
      **NOTE:** This service been marked for deprecation: 
https://developer.github.com/changes/2018-04-25-github-services-deprecation/

      Functionality will be removed from GitHub.com on January 31st, 2019.