[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-block] [PATCH v5 00/15] qcow2: space preallocation and COW improve
From: |
Anton Nefedov |
Subject: |
[Qemu-block] [PATCH v5 00/15] qcow2: space preallocation and COW improvements |
Date: |
Wed, 1 Nov 2017 18:43:53 +0300 |
v5: rebased, patches slightly reordered;
cluster allocation (to avoid writing zero-buffers)
now goes before an optional ahead preallocation
v4: http://lists.nongnu.org/archive/html/qemu-devel/2017-08/msg00109.html
This pull request is to improve a few performance problems of qcow2 format:
1. non cluster-aligned write requests (to unallocated clusters) explicitly
pad data with zeroes if there is no backing data.
Resulting increase in ops number and potential cluster fragmentation
(on the host file) is already solved by:
ee22a9d qcow2: Merge the writing of the COW regions with the guest data
However, in case of zero COW regions, that can be avoided at all
but the whole clusters are preallocated and zeroed in a single
efficient write_zeroes() operation
2. moreover, efficient write_zeroes() operation can be used to preallocate
space megabytes (*configurable number) ahead which gives noticeable
improvement on some storage types (e.g. distributed storage)
where the space allocation operation might be expensive)
3. this will also allow to enable simultaneous writes to the same unallocated
cluster after the space has been allocated & zeroed but before
the first data is written and the cluster is linked to L2.
(Not included in this patchset).
Efficient write_zeroes usually implies that the blocks are not actually
written to but only reserved and marked as zeroed by the storage.
In this patchset, file-posix driver is marked as supporting this operation
if it supports (/configured to support) fallocate() operation.
Existing bdrv_write_zeroes() falls back to writing zero buffers if
write_zeroes is not supported by the driver.
A new flag (BDRV_REQ_ALLOCATE) is introduced to avoid that but return ENOTSUP.
Such allocate requests are also implemented to possibly overlap with the
other requests. No wait is performed but an error returned in such case as well.
So the operation should be considered advisory and a fallback scenario still
handled by the caller (in this case, qcow2 driver).
Anton Nefedov (12):
mirror: inherit supported write/zero flags
blkverify: set supported write/zero flags
block: introduce BDRV_REQ_ALLOCATE flag
block: treat BDRV_REQ_ALLOCATE as serialising
file-posix: support BDRV_REQ_ALLOCATE
block: support BDRV_REQ_ALLOCATE in passthrough drivers
qcow2: move is_zero() up
qcow2: skip writing zero buffers to empty COW areas
qcow2: set inactive flag
qcow2: do not zero out clusters if already preallocated
iotest 198: test BDRV_REQ_ALLOCATE
iotest 134: test cluster-misaligned encrypted write
Denis V. Lunev (2):
qcow2: preallocation at image expand
qcow2: truncate preallocated space
Pavel Butsykin (1):
qcow2: check space leak at the end of the image
block/qcow2.h | 18 ++++
include/block/block.h | 6 +-
include/block/block_int.h | 2 +-
block/blkdebug.c | 3 +-
block/blkverify.c | 9 ++
block/file-posix.c | 8 ++
block/io.c | 47 +++++++--
block/mirror.c | 5 +
block/qcow2-cluster.c | 14 ++-
block/qcow2-refcount.c | 7 ++
block/qcow2.c | 202 +++++++++++++++++++++++++++++++++----
block/raw-format.c | 3 +-
block/trace-events | 1 +
qemu-options.hx | 4 +
tests/qemu-iotests/026.out | 94 +++++++++++++----
tests/qemu-iotests/026.out.nocache | 94 +++++++++++++----
tests/qemu-iotests/029.out | 5 +-
tests/qemu-iotests/060 | 2 +-
tests/qemu-iotests/060.out | 13 ++-
tests/qemu-iotests/061.out | 5 +-
tests/qemu-iotests/066 | 2 +-
tests/qemu-iotests/066.out | 9 +-
tests/qemu-iotests/098.out | 7 +-
tests/qemu-iotests/108.out | 5 +-
tests/qemu-iotests/112.out | 5 +-
tests/qemu-iotests/134 | 9 ++
tests/qemu-iotests/134.out | 10 ++
tests/qemu-iotests/198 | 146 +++++++++++++++++++++++++++
tests/qemu-iotests/198.out | 50 +++++++++
tests/qemu-iotests/group | 1 +
30 files changed, 693 insertions(+), 93 deletions(-)
create mode 100755 tests/qemu-iotests/198
create mode 100644 tests/qemu-iotests/198.out
--
2.7.4
- [Qemu-block] [PATCH v5 00/15] qcow2: space preallocation and COW improvements,
Anton Nefedov <=
- [Qemu-block] [PATCH v5 01/15] mirror: inherit supported write/zero flags, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 02/15] blkverify: set supported write/zero flags, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 07/15] qcow2: move is_zero() up, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 04/15] block: treat BDRV_REQ_ALLOCATE as serialising, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 10/15] qcow2: set inactive flag, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 06/15] block: support BDRV_REQ_ALLOCATE in passthrough drivers, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 05/15] file-posix: support BDRV_REQ_ALLOCATE, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 08/15] qcow2: skip writing zero buffers to empty COW areas, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 03/15] block: introduce BDRV_REQ_ALLOCATE flag, Anton Nefedov, 2017/11/01
- [Qemu-block] [PATCH v5 14/15] iotest 198: test BDRV_REQ_ALLOCATE, Anton Nefedov, 2017/11/01