[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Qemu-block] [PULL 26/42] drive-backup: added support for data compressi
From: |
Kevin Wolf |
Subject: |
[Qemu-block] [PULL 26/42] drive-backup: added support for data compression |
Date: |
Mon, 5 Sep 2016 20:13:38 +0200 |
From: Pavel Butsykin <address@hidden>
The idea is simple - backup is "written-once" data. It is written block
by block and it is large enough. It would be nice to save storage
space and compress it.
The patch adds a flag to the qmp/hmp drive-backup command which enables
block compression. Compression should be implemented in the format driver
to enable this feature.
There are some limitations of the format driver to allow compressed writes.
We can write data only once. Though for backup this is perfectly fine.
These limitations are maintained by the driver and the error will be
reported if we are doing something wrong.
Signed-off-by: Pavel Butsykin <address@hidden>
Reviewed-by: Stefan Hajnoczi <address@hidden>
Signed-off-by: Denis V. Lunev <address@hidden>
CC: Jeff Cody <address@hidden>
CC: Markus Armbruster <address@hidden>
CC: Eric Blake <address@hidden>
CC: John Snow <address@hidden>
CC: Stefan Hajnoczi <address@hidden>
CC: Kevin Wolf <address@hidden>
Signed-off-by: Kevin Wolf <address@hidden>
---
block/backup.c | 12 +++++++++++-
blockdev.c | 9 ++++++---
hmp-commands.hx | 8 +++++---
hmp.c | 3 +++
include/block/block_int.h | 1 +
qapi/block-core.json | 5 ++++-
qmp-commands.hx | 5 ++++-
7 files changed, 34 insertions(+), 9 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 2c05323..bb3bb9a 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -47,6 +47,7 @@ typedef struct BackupBlockJob {
uint64_t sectors_read;
unsigned long *done_bitmap;
int64_t cluster_size;
+ bool compress;
NotifierWithReturn before_write;
QLIST_HEAD(, CowRequest) inflight_reqs;
} BackupBlockJob;
@@ -154,7 +155,8 @@ static int coroutine_fn backup_do_cow(BackupBlockJob *job,
bounce_qiov.size, BDRV_REQ_MAY_UNMAP);
} else {
ret = blk_co_pwritev(job->target, start * job->cluster_size,
- bounce_qiov.size, &bounce_qiov, 0);
+ bounce_qiov.size, &bounce_qiov,
+ job->compress ? BDRV_REQ_WRITE_COMPRESSED :
0);
}
if (ret < 0) {
trace_backup_do_cow_write_fail(job, start, ret);
@@ -477,6 +479,7 @@ static void coroutine_fn backup_run(void *opaque)
void backup_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
+ bool compress,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
BlockCompletionFunc *cb, void *opaque,
@@ -507,6 +510,12 @@ void backup_start(const char *job_id, BlockDriverState *bs,
return;
}
+ if (compress && target->drv->bdrv_co_pwritev_compressed == NULL) {
+ error_setg(errp, "Compression is not supported for this drive %s",
+ bdrv_get_device_name(target));
+ return;
+ }
+
if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
return;
}
@@ -555,6 +564,7 @@ void backup_start(const char *job_id, BlockDriverState *bs,
job->sync_mode = sync_mode;
job->sync_bitmap = sync_mode == MIRROR_SYNC_MODE_INCREMENTAL ?
sync_bitmap : NULL;
+ job->compress = compress;
/* If there is no backing file on the target, we cannot rely on COW if our
* backup cluster size is smaller than the target cluster size. Even for
diff --git a/blockdev.c b/blockdev.c
index 5f1b015..f5857d0 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3132,6 +3132,9 @@ static void do_drive_backup(DriveBackup *backup,
BlockJobTxn *txn, Error **errp)
if (!backup->has_job_id) {
backup->job_id = NULL;
}
+ if (!backup->has_compress) {
+ backup->compress = false;
+ }
bs = qmp_get_root_bs(backup->device, errp);
if (!bs) {
@@ -3210,8 +3213,8 @@ static void do_drive_backup(DriveBackup *backup,
BlockJobTxn *txn, Error **errp)
}
backup_start(backup->job_id, bs, target_bs, backup->speed, backup->sync,
- bmap, backup->on_source_error, backup->on_target_error,
- block_job_cb, bs, txn, &local_err);
+ bmap, backup->compress, backup->on_source_error,
+ backup->on_target_error, block_job_cb, bs, txn, &local_err);
bdrv_unref(target_bs);
if (local_err != NULL) {
error_propagate(errp, local_err);
@@ -3277,7 +3280,7 @@ void do_blockdev_backup(BlockdevBackup *backup,
BlockJobTxn *txn, Error **errp)
}
}
backup_start(backup->job_id, bs, target_bs, backup->speed, backup->sync,
- NULL, backup->on_source_error, backup->on_target_error,
+ NULL, false, backup->on_source_error, backup->on_target_error,
block_job_cb, bs, txn, &local_err);
if (local_err != NULL) {
error_propagate(errp, local_err);
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 848efee..74f32e5 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1182,8 +1182,8 @@ ETEXI
{
.name = "drive_backup",
- .args_type = "reuse:-n,full:-f,device:B,target:s,format:s?",
- .params = "[-n] [-f] device target [format]",
+ .args_type =
"reuse:-n,full:-f,compress:-c,device:B,target:s,format:s?",
+ .params = "[-n] [-f] [-c] device target [format]",
.help = "initiates a point-in-time\n\t\t\t"
"copy for a device. The device's contents are\n\t\t\t"
"copied to the new image file, excluding data
that\n\t\t\t"
@@ -1191,7 +1191,9 @@ ETEXI
"The -n flag requests QEMU to reuse the image
found\n\t\t\t"
"in new-image-file, instead of recreating it from
scratch.\n\t\t\t"
"The -f flag requests QEMU to copy the whole
disk,\n\t\t\t"
- "so that the result does not need a backing
file.\n\t\t\t",
+ "so that the result does not need a backing
file.\n\t\t\t"
+ "The -c flag requests QEMU to compress backup
data\n\t\t\t"
+ "(if the target format supports it).\n\t\t\t",
.mhandler.cmd = hmp_drive_backup,
},
STEXI
diff --git a/hmp.c b/hmp.c
index 3c06200..a7dfe6f 100644
--- a/hmp.c
+++ b/hmp.c
@@ -1109,6 +1109,7 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict)
const char *format = qdict_get_try_str(qdict, "format");
bool reuse = qdict_get_try_bool(qdict, "reuse", false);
bool full = qdict_get_try_bool(qdict, "full", false);
+ bool compress = qdict_get_try_bool(qdict, "compress", false);
Error *err = NULL;
DriveBackup backup = {
.device = (char *)device,
@@ -1118,6 +1119,8 @@ void hmp_drive_backup(Monitor *mon, const QDict *qdict)
.sync = full ? MIRROR_SYNC_MODE_FULL : MIRROR_SYNC_MODE_TOP,
.has_mode = true,
.mode = reuse ? NEW_IMAGE_MODE_EXISTING :
NEW_IMAGE_MODE_ABSOLUTE_PATHS,
+ .has_compress = !!compress,
+ .compress = compress,
};
if (!filename) {
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 9c61f49..0ca6a78 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -765,6 +765,7 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
void backup_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, int64_t speed,
MirrorSyncMode sync_mode, BdrvDirtyBitmap *sync_bitmap,
+ bool compress,
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
BlockCompletionFunc *cb, void *opaque,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 4c8cf05..300a68b 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -898,6 +898,9 @@
# Must be present if sync is "incremental", must NOT be present
# otherwise. (Since 2.4)
#
+# @compress: #optional true to compress data, if the target format supports it.
+# (default: false) (since 2.7)
+#
# @on-source-error: #optional the action to take on an error on the source,
# default 'report'. 'stop' and 'enospc' can only be used
# if the block device supports io-status (see BlockInfo).
@@ -915,7 +918,7 @@
{ 'struct': 'DriveBackup',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*format': 'str', 'sync': 'MirrorSyncMode', '*mode':
'NewImageMode',
- '*speed': 'int', '*bitmap': 'str',
+ '*speed': 'int', '*bitmap': 'str', '*compress': 'bool',
'*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError' } }
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 956f1b0..d1593c9 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1217,7 +1217,8 @@ EQMP
{
.name = "drive-backup",
.args_type = "job-id:s?,sync:s,device:B,target:s,speed:i?,mode:s?,"
-
"format:s?,bitmap:s?,on-source-error:s?,on-target-error:s?",
+ "format:s?,bitmap:s?,compress:b?,"
+ "on-source-error:s?,on-target-error:s?",
.mhandler.cmd_new = qmp_marshal_drive_backup,
},
@@ -1253,6 +1254,8 @@ Arguments:
- "mode": whether and how QEMU should create a new image
(NewImageMode, optional, default 'absolute-paths')
- "speed": the maximum speed, in bytes per second (json-int, optional)
+- "compress": true to compress data, if the target format supports it.
+ (json-bool, optional, default false)
- "on-source-error": the action to take on an error on the source, default
'report'. 'stop' and 'enospc' can only be used
if the block device supports io-status.
--
1.8.3.1
- [Qemu-block] [PULL 17/42] qcow2: add qcow2_co_pwritev_compressed, (continued)
- [Qemu-block] [PULL 17/42] qcow2: add qcow2_co_pwritev_compressed, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 18/42] qcow2: cleanup qcow2_co_pwritev_compressed to avoid the recursion, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 21/42] qcow: cleanup qcow_co_pwritev_compressed to avoid the recursion, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 20/42] qcow: add qcow_co_pwritev_compressed, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 22/42] block: remove BlockDriver.bdrv_write_compressed, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 28/42] qemu-iotests: test backup compression in 055, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 23/42] block/io: turn on dirty_bitmaps for the compressed writes, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 24/42] block: simplify drive-backup, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 25/42] block: simplify blockdev-backup, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 29/42] qemu-iotests: add vmdk for test backup compression in 055, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 26/42] drive-backup: added support for data compression,
Kevin Wolf <=
- [Qemu-block] [PULL 27/42] blockdev-backup: added support for data compression, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 30/42] test-coroutine: Fix coroutine pool corruption, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 32/42] coroutine: Let CoMutex remember who holds it, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 31/42] qcow2: fix iovec size at qcow2_co_pwritev_compressed, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 34/42] block jobs: Improve error message for missing job ID, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 35/42] qemu-iotests: Log QMP traffic in debug mode, Kevin Wolf, 2016/09/05
- [Qemu-block] [PULL 36/42] block: Allow node name for 'qemu-io' HMP command, Kevin Wolf, 2016/09/05