[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH v22 3/4] qcow2: add zstd cluster compression
From: |
Max Reitz |
Subject: |
Re: [PATCH v22 3/4] qcow2: add zstd cluster compression |
Date: |
Wed, 29 Apr 2020 14:17:05 +0200 |
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.7.0 |
On 29.04.20 12:37, Vladimir Sementsov-Ogievskiy wrote:
> 29.04.2020 13:24, Max Reitz wrote:
>> On 28.04.20 22:00, Denis Plotnikov wrote:
>>> zstd significantly reduces cluster compression time.
>>> It provides better compression performance maintaining
>>> the same level of the compression ratio in comparison with
>>> zlib, which, at the moment, is the only compression
>>> method available.
>>>
>>> The performance test results:
>>> Test compresses and decompresses qemu qcow2 image with just
>>> installed rhel-7.6 guest.
>>> Image cluster size: 64K. Image on disk size: 2.2G
>>>
>>> The test was conducted with brd disk to reduce the influence
>>> of disk subsystem to the test results.
>>> The results is given in seconds.
>>>
>>> compress cmd:
>>> time ./qemu-img convert -O qcow2 -c -o compression_type=[zlib|zstd]
>>> src.img [zlib|zstd]_compressed.img
>>> decompress cmd
>>> time ./qemu-img convert -O qcow2
>>> [zlib|zstd]_compressed.img uncompressed.img
>>>
>>> compression decompression
>>> zlib zstd zlib zstd
>>> ------------------------------------------------------------
>>> real 65.5 16.3 (-75 %) 1.9 1.6 (-16 %)
>>> user 65.0 15.8 5.3 2.5
>>> sys 3.3 0.2 2.0 2.0
>>>
>>> Both ZLIB and ZSTD gave the same compression ratio: 1.57
>>> compressed image size in both cases: 1.4G
>>>
>>> Signed-off-by: Denis Plotnikov <address@hidden>
>>> QAPI part:
>>> Acked-by: Markus Armbruster <address@hidden>
>>> ---
>>> docs/interop/qcow2.txt | 1 +
>>> configure | 2 +-
>>> qapi/block-core.json | 3 +-
>>> block/qcow2-threads.c | 169 +++++++++++++++++++++++++++++++++++++++++
>>> block/qcow2.c | 7 ++
>>> slirp | 2 +-
>>> 6 files changed, 181 insertions(+), 3 deletions(-)
>>
>> [...]
>>
>>> diff --git a/block/qcow2-threads.c b/block/qcow2-threads.c
>>> index 7dbaf53489..a0b12e1b15 100644
>>> --- a/block/qcow2-threads.c
>>> +++ b/block/qcow2-threads.c
>>
>> [...]
>>
>>> +static ssize_t qcow2_zstd_decompress(void *dest, size_t dest_size,
>>> + const void *src, size_t src_size)
>>> +{
>>
>> [...]
>>
>>> + /*
>>> + * The compressed stream from the input buffer may consist of more
>>> + * than one zstd frame.
>>
>> Can it?
>
> If not, we must require it in the specification.
Actually, now that you mention it, it would make sense anyway to add
some note to the specification on what exactly compressed with zstd means.
> Hmm. If at some point
> we'll want multi-threaded compression of one big (2M) cluster.. Could
> this be implemented with zstd lib, if multiple frames are allowed, will
> allowing multiple frames help? I don't know actually, but I think better
> not to forbid it. On the other hand, I don't see any benefit in large
> compressed clusters. At least, in our scenarios (for compressed backups)
> we use 64k compressed clusters, for good granularity of incremental
> backups (when for running vm we use 1M clusters).
Is it really that important? Naïvely, it sounds rather complicated to
introduce multithreading into block drivers.
(Also, as for compression, it can only be used in backup scenarios
anyway, where you write many clusters at once. So parallelism on the
cluster level should sufficient to get high usage, and it would benefit
all compression types and cluster sizes.)
Max
signature.asc
Description: OpenPGP digital signature
- [PATCH v22 0/4] implement zstd cluster compression method, Denis Plotnikov, 2020/04/28
- [PATCH v22 3/4] qcow2: add zstd cluster compression, Denis Plotnikov, 2020/04/28
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Eric Blake, 2020/04/28
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Max Reitz, 2020/04/29
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Vladimir Sementsov-Ogievskiy, 2020/04/29
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression,
Max Reitz <=
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Vladimir Sementsov-Ogievskiy, 2020/04/29
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Eric Blake, 2020/04/29
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Max Reitz, 2020/04/30
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Denis Plotnikov, 2020/04/30
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Max Reitz, 2020/04/30
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Denis Plotnikov, 2020/04/30
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Denis Plotnikov, 2020/04/29
- Re: [PATCH v22 3/4] qcow2: add zstd cluster compression, Max Reitz, 2020/04/29
[PATCH v22 2/4] qcow2: rework the cluster compression routine, Denis Plotnikov, 2020/04/28
[PATCH v22 1/4] qcow2: introduce compression type feature, Denis Plotnikov, 2020/04/28