Re: [Qemu-devel] [PATCH 1/6 v11] docs: spec for add-cow file format

[Stefan Hajnoczi]

On Tue, Jul 31, 2012 at 5:51 PM, Dong Xu Wang
<address@hidden> wrote:
> Introduce a new file format:add-cow. The usage can be found at this patch.
>
> Signed-off-by: Dong Xu Wang <address@hidden>
> ---
> Now add-cow is still using QEMUOptionParameter, not QemuOpts,  I will send a
> seperate patch series to convert.

I suggest including a cover letter in future patch series:

git format-patch --cover-letter --numbered -o my-series/ master..

Or to do it in a single command with git-send-email(1) use the --compose option.

The cover letter makes it easy for reviewers to add their Reviewed-by:
to the entire series by replying only once.  It's also a good place to
include a changelog that tells reviewers what you changed from the
last published version.

>  docs/specs/add-cow.txt |  128 
> ++++++++++++++++++++++++++++++++++++++++++++++++
>  1 files changed, 128 insertions(+), 0 deletions(-)
>  create mode 100644 docs/specs/add-cow.txt
>
> diff --git a/docs/specs/add-cow.txt b/docs/specs/add-cow.txt
> new file mode 100644
> index 0000000..4793a3e
> --- /dev/null
> +++ b/docs/specs/add-cow.txt
> @@ -0,0 +1,128 @@
> +== General ==
> +
> +Raw file format does not support backing file and copy on write feature.
> +The add-cow image format makes it possible to use backing files with raw
> +image by keeping a separate .add-cow metadata file. Once all sectors
> +have been written into the raw image it is safe to discard the .add-cow
> +and backing files, then we can use the raw image directly.
> +
> +While using add-cow, procedures may like this:
> +(ubuntu.img is a disk image which has been installed OS.)
> +    1)  Create a raw image with the same size of ubuntu.img
> +            qemu-img create -f raw test.raw 8G
> +    2)  Create an add-cow image which will store dirty bitmap
> +            qemu-img create -f add-cow test.add-cow \
> +                -o backing_file=ubuntu.img,image_file=test.raw
> +    3)  Run qemu with add-cow image
> +            qemu -drive if=virtio,file=test.add-cow
> +
> +test.raw may be larger than ubuntu.img, in that case, the size of 
> test.add-cow
> +will be calculated by the size of ubuntu.img, test.raw will be used from the
> +1st byte, the rest part can be used for other purpose.

This is not how backing files normally work.  With qcow2 or qed a
smaller backing file just means that the guest reads zeroes from the
areas beyond the end of the backing file.  Is there a special reason
why you want to implement the behavior you described in the spec?
Otherwise I suggest implementing the same behavior as qcow2/qed.

> +
> +=Specification=
> +
> +The file format looks like this:
> +
> + +---------------+-------------+-----------------+
> + |     Header    |   Reserved  |    COW bitmap   |
> + +---------------+-------------+-----------------+
> +
> +All numbers in add-cow are stored in Little Endian byte order.
> +
> +== Header ==
> +
> +The Header is included in the first bytes:
> +(#define HEADER_SIZE (4096 * header_pages_size))
> +    Byte    0 -  7:     magic
> +                        add-cow magic string ("ADD_COW\xff").
> +
> +            8 -  11:    version
> +                        Version number (only valid value is 1 now).
> +
> +            12 - 15:    backing file name offset
> +                        Offset in the add-cow file at which the backing file
> +                        name is stored (NB: The string is not null 
> terminated).
> +                        If backing file name does NOT exist, this field will 
> be
> +                        0. Must be between 80 and [HEADER_SIZE - 2](a file 
> name
> +                        must be at least 1 byte).
> +
> +            16 - 19:    backing file name size
> +                        Length of the backing file name in bytes. It will be > 0
> +                        if the backing file name offset is 0. If backing file
> +                        name offset is non-zero, then it must be non-zero. 
> Must
> +                        be less than [HEADER_SIZE - 80] to fit in the 
> reserved
> +                        part of the header.
> +
> +            20 - 23:    image file name offset
> +                        Offset in the add-cow file at which the image file 
> name
> +                        is stored (NB: The string is not null terminated). It
> +                        must be between 80 and [HEADER_SIZE - 2].
> +
> +            24 - 27:    image file name size
> +                        Length of the image file name in bytes.
> +                        Must be less than [HEADER_SIZE - 80] to fit in the 
> reserved
> +                        part of the header.
> +
> +            28 - 35:    features
> +                        Currently only 3 feature bit is used:
> +                        Feature bits:
> +                            The image uses a backing file:
> +                            * ADD_COW_F_BACKING_FILE    = 0x01.
> +                            The image uses a image file:
> +                            * ADD_COW_F_IMAGE_FILE      = 0x02.
> +                            All bits in bitmap have been set to 1, add-cow 
> wrapper
> +                            can be discarded.
> +                            * ADD_COW_F_All_ALLOCATED   = 0x04.
> +
> +            36 - 43:    optional features
> +                        Not used now. Researved for future use.

s/Researved/Reserved/

> +
> +            44 - 47:    header pages size
> +                        The header field is variable-sized. This field 
> indicates
> +                        how many pages(4k) will be used to store add-cow 
> header.
> +                        In add-cow v1, it is fixed to 1, so the header size 
> will
> +                        be 4k * 1 = 4096 bytes.
> +
> +Image file name and backing file name must NOT be the same, we prevent this
> +while creating add-cow files.
> +
> +Image file and backing file are interpreted relative to the qcow2 file, not
> +to the current working directory of the process that opened the qcow2 file.
> +
> +== Reserved ==
> +
> +    Byte    48 - 63:    backing file format
> +                        format of backing file. It will be filled with 0 if
> +                        backing file name offset is 0. If backing file name
> +                        offset is none-zero, it must be non-zero.

s/none-zero/non-zero/

> +
> +            64 - 79:    image file format
> +                        format of image file. It must be non-zero.
> +
> +            80 - [HEADER_SIZE - 1]:
> +                        It is used to make sure COW bitmap field starts at 
> the
> +                        HEADER_SIZE byte, backing file name and image file 
> name
> +                        will be stored here.
> +
> +== COW bitmap ==
> +
> +The "COW bitmap" field starts at the 4096th byte, stores a bitmap related to

I would say it starts at offset HEADER_SIZE.  4096th byte == offset
0x0fff, 4097th byte == offset 0x1000.

Stefan