qemu-block
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-block] QCOW2 support for LZO compression


From: Kevin Wolf
Subject: Re: [Qemu-block] QCOW2 support for LZO compression
Date: Mon, 26 Jun 2017 13:21:12 +0200
User-agent: Mutt/1.5.21 (2010-09-15)

Am 26.06.2017 um 12:08 hat Peter Lieven geschrieben:
> Am 26.06.2017 um 11:57 schrieb Kevin Wolf:
> >Am 26.06.2017 um 11:20 hat Peter Lieven geschrieben:
> >>>So he chose a different algorithm (zstd). When I asked, he posted a
> >>>comparison of algorithms (however a generic one and not measured in the
> >>>context of qemu) that suggests that LZO would be slightly faster, but
> >>>have a considerable worse compression ratio with the settings that were
> >>>benchmarked.
> >>My idea to choose LZO was that it is widely available and available in
> >>any distro you can think of. We already have probing for it in configure.
> >>My concern with ZSTD would be that it seems there are no packages
> >>available for most distros and that it seems to be multi-threaded. I don't
> >>know if this will cause any trouble?
> >The availability and that we already link against LZO is a good point. I
> >think we want to avoid a situation where compressed qcow2 files can't be
> >read by binaries of popular distributions - after all, downloadable
> >images are an important use case for compressed images.
> 
> As long as the default remains gzip I don't see any issues. If you choose
> a different algorithm, you should know what you are doing.

If we add a better compression algorithm, then it's clear to me that the
goal is to make it the default in a year or two.

> >>>I think it's clear that if there is any serious interest in compression,
> >>>we'll want to support at least one more algorithm. What we still need to
> >>>evaluate is which one(s) to take, and whether a simple incompatible flag
> >>>in the header like in Den's patch is enough or whether we should add a
> >>>whole new header field for the compression algorithm (like we already
> >>>have for encryption).
> >> From my side there clearly is interest in optimizing the compression. Its
> >>even possible to speed up zlib by 3-4x times by choosing other parameters
> >>for deflate which unfortunately are not compatible with our inflate 
> >>settings.
> >>
> >>I don't know if its worth creating a new header field. Even if we spent to 
> >>bits
> >>in the end (one for LZO and one for ZSDT). I think this wouldn't hurt. 
> >>However,
> >>there are likely to pop up new compression algorithms in the future and
> >>a header would be more flexible.
> >That we could just use different parameters is an important hint that
> >maybe we don't just need bits to select an algorithm, but also some
> >space for parameters for the selected algorithm. In this case I think we
> >need a header change anyway.
> 
> The issue is, you don't know in advance how much space will be needed
> for the paramters.  Maybe it would be simpler to have different
> settings for a compression algorithm threated as different algorithms.
> Like zstd-fast, zstd, zstd-slow, gzip, lzo, lzo-slow etc.

Yes, how much space is needed for parameters depends on the algorithm.
Normally, when adding an incompatible feature, you would put the new
options inside the main header. However, nothing stops us from using a
header extension instead like we would with compatible features.

So I think this is what we could do:

 * Add a header extension type per algorithm that contains the
   parameters that are needed for this parameter

 * Add a header field that selects the compression algorithm (e.g. a
   32 bit field containing the matching header extension ID). Images
   that contain compression header extensions that don't match this
   header field (or images containing the same compression header
   extension type more than once) are rejected

 * Add an incompatible feature flag that indicates presence of the
   header field + header extension

> >>I just don't want to make it too complicated and as you pointed out
> >>compression is not that interesting for most people - maybe due to its
> >>speed.
> >In fact, I think it could be the poor integration in qemu. Compressed
> >images might be more appealing if you couldn't only write compressed
> >data with special-case operations like 'qemu-img convert', but just give
> >an option that you want all writes to be compressed and then use it like
> >any other image.
> 
> Then we should start to improve that.

Thanks for volunteering. ;-)

Kevin



reply via email to

[Prev in Thread] Current Thread [Next in Thread]