On Mon, Jun 26, 2017 at 11:20:33AM +0200, Peter Lieven wrote:
Am 26.06.2017 um 10:28 schrieb Kevin Wolf:
[ Cc: qemu-devel; don't post to qemu-block only! ]
Am 26.06.2017 um 09:57 hat Peter Lieven geschrieben:
Hi,
I am currently working on optimizing speed for compressed QCOW2
images. We use them for templates and would also like to use them for
backups, but the latter is almost infeasible because using gzip for
compression is horribly slow. I tried to experiment with different
options to deflate, but in the end I think its better to use a
different compression algorithm for cases where speed matters. As we
already have probing for it in configure and as it is widely used I
would like to use LZO for that purpose. I think it would be best to
have a flag to indicate that compressed blocks use LZO compression,
but I would need a little explaination which of the feature fields I
have to use to prevent an older (incompatible) Qemu opening LZO
compressed QCOW2 images.
I also have already some numbers. I converted a fresh Debian 9 Install
which has an uncomressed QCOW2 size of 1158 MB with qemu-img to a
compressed QCOW2. With GZIP compression the result is 356MB whereas
the LZO version is 452MB. However, the current GZIP variant uses 35
seconds for this operation where LZO only needs 4 seconds. I think is
is a good trade in especially when its optional so the user can
choose.
What are your thoughts?
We had a related RFC patch by Den earlier this year, which never
received many comment and never got out of RFC:
https://lists.gnu.org/archive/html/qemu-devel/2017-03/msg04682.html
I was not aware of that one. Thanks for pointing out.
So he chose a different algorithm (zstd). When I asked, he posted a
comparison of algorithms (however a generic one and not measured in the
context of qemu) that suggests that LZO would be slightly faster, but
have a considerable worse compression ratio with the settings that were
benchmarked.
My idea to choose LZO was that it is widely available and available in
any distro you can think of. We already have probing for it in configure.
My concern with ZSTD would be that it seems there are no packages
available for most distros and that it seems to be multi-threaded. I don't
know if this will cause any trouble?
As a distro maintainer I'd always prefer option to use a library that is
already widely available. While ZSTD could certainly be added to distros,
it means the QEMU maintainer will end up having to package it & become
the defacto long term maintainer of it long term, which is an extra burden.
WRT to making compression algorithms configurable, I think it is important
to ensure we don't add lots of optional algorithms. An important factor is
portability of images - we don't want to end up with each distro's build
of QEMU enabling a different sub-set of compression algorithms, as that is
going to harm interoperability for distributed images. This again makes me
prefer a compression format whose library is widely available, as that makes
it highly likely that the distro will choose to enable it during build.
Regards,
Daniel