[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Qemu-discuss] disk image: self-organized format or raw file

From: Jakob Bohm
Subject: Re: [Qemu-discuss] disk image: self-organized format or raw file
Date: Tue, 12 Aug 2014 14:02:03 +0200
User-agent: Mozilla/5.0 (Windows NT 5.2; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.0

On 8/12/2014 1:51 AM, 吴兴博 wrote:

   The introduction in the wiki page present several advantages of qcow2
[1]. But I'm a little confused. I really appreciate if any one can give
me some help on this :).

  (1) Currently the raw format doesn't support COW. In other words, a
raw image cannot have a backing file. COW depends on the mapping table
on which we it knows whether each block/cluster is present (has been
modified) in the current image file. Modern file-systems like
xfs/ext4/etc. provide extent/block allocation information to user-level.
Like what 'filefrag' does with ioctl 'FIBMAP' and 'FIEMAP'. I guess the
raw file driver (maybe block/raw-posix.c) may obtain correct 'present
information about blocks. However this information may be limited to be
aligned with file allocation unit size. Maybe it's just because a raw
file has no space to store the "backing file name"? I don't think this
could hinder the useful feature.

The ability to store a backing file name or other per-image metadata
(such as emulated disk geometry) is not that critical, as this could
simply be provided in a config file or command line option.  In fact
there are scenarios where the storage of this data inside the diskimage
files is actually a problem.

However the present/not-present feature of "sparse" files (on almost any
file system) is subject to the rule that both the file system and basic
file utilities (/bin/cp, /bin/tar etc. etc.) are allowed to arbitrarily
convert byte ranges back and forth between all-zeroes and "bytes
absent".  This in term means that there will be no reliable way for qemu
etc. to distinguish between a hole that should be mapped to the content
of the next cow level down and byte ranges where non-zero bytes in the
next cow level down have been overridden by bytes with the value zero.

Thus to reliably maintain a COW overlay which lists some byte ranges
as being "whatever the next level closer to the base file says" and
others as being "some value (maybe zero) stored in the cow overlay".

All the various COW file formats supported by qemu have been designed
as different solutions to this requirement.  qcow2 is the "native"
qemu format and as such is supposed to always have the most complete
feature set and the most optimised and tested implementation details.

  (2) As most popular filesystems support delay-allocation/on-demand
allocation/holes, whatever, a raw image is also thin provisioned as
other formats. It doesn't consume much disk space by storing useless
zeros. However, I don't know if there is any concern on whether
fragmented extents would become a burden of the host filesystem.

A much more significant issue is the lack of a standard POSIX API for
changing a byte range in a file from "non-hole allocated space" to
"sparse hole with no allocated disk space".  This makes it hard to
release disk space previously consumed by the guest filesystem.  When
a COW file format is used, that key opearation is now an internal
detail independent of the underlying host file system, host kernel
version etc.

  (3) For compression and encryption, I'm not an export on these topics
at all but I think these features may not be vital to a image format as
both guest/host's filesystem can also provide similar functionality.

I agree with you there.

  (4) I don't have too much understanding on how snapshot works but I
think theoretically it would be using the techniques no more than that
used in COW and backing file.

A full featured snapshot (as supported by qcow2 and several non-qemu VM
systems) consists of two things representing a single point in time: A
split between a (possible itself COW) base image representing the
virtual disk content at time T and a COW overlay representing any later
changes to the virtual disk content.  AND a stored copy of the virtual
physical memory and virtual hardware device states at time T.  Together
these two items allow a virtual machine to be "instantaneously"
reverted to whatever it was at time T and continue running from that
point as if nothing had happened.

After all these thoughts, I still found no reason to not using a 'raw'
file image (engineering efforts in Qemu should not count as we don't ask
  for more features from outside world).
I would be very sorry if my ignorance wasted your time.

[1] http://en.wikibooks.org/wiki/QEMU/Images#Image_types
"QEMU supports several image types. The "native" and most flexible type
is /qcow2/, which supports copy on write
<http://en.wikibooks.org/wiki/QEMU/Images#Copy_on_write>, encryption,
compression, and VM snapshots.

        吴兴博  Wu, Xingbo <address@hidden <mailto:address@hidden>>


Jakob Bohm, CIO, Partner, WiseMo A/S.  http://www.wisemo.com
Transformervej 29, 2730 Herlev, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded

reply via email to

[Prev in Thread] Current Thread [Next in Thread]