[Qemu-devel] [PATCH v2] persistent dirty bitmap: add QDB file spec.

[Vladimir Sementsov-Ogievskiy]

QDB file is for storing dirty bitmap. The specification is based on
qcow2 specification.

Saving several bitmaps is necessary when server shutdowns during
backup. In this case 2 tables for each disk are available. One
collected for a previous period and one active. Though this feature
is discussable.

Big endian format and Standard Cluster Descriptor are used to simplify
integration with qcow2, to support internal bitmaps for qcow2 in future.

The idea is that the same procedure writing the data to QDB file could
do the same for QCOW2. The only difference is cluster refcount table.
Should we use it here or not is still questionable.

Signed-off-by: Vladimir Sementsov-Ogievskiy <address@hidden>
---
 docs/specs/qdb.txt | 132 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 132 insertions(+)
 create mode 100644 docs/specs/qdb.txt

diff --git a/docs/specs/qdb.txt b/docs/specs/qdb.txt
new file mode 100644
index 0000000..d570a69
--- /dev/null
+++ b/docs/specs/qdb.txt
@@ -0,0 +1,132 @@
+== General ==
+
+"QDB" means "Qemu Dirty Bitmaps". QDB file can store several dirty bitmaps.
+QDB file is organized in units of constant size, which are called clusters.
+
+All numbers in QDB are stored in Big Endian byte order.
+
+== Header ==
+
+The first cluster of a QDB image contains the file header:
+
+    Byte  0 -  3:   magic
+                    QDB magic string ("QDB\0")
+
+          4 -  7:   version
+                    Version number (valid value is 1)
+
+          8 - 11:   cluster_bits
+                    Number of bits that are used for addressing an offset
+                    within a cluster (1 << cluster_bits is the cluster size).
+                    Must not be less than 9 (i.e. 512 byte clusters).
+
+         12 - 15:   nb_bitmaps
+                    Number of bitmaps contained in the file
+
+         16 - 23:   bitmaps_offset
+                    Offset into the QDB file at which the bitmap table starts.
+                    Must be aligned to a cluster boundary.
+
+         24 - 27:   header_length
+                    Length of the header structure in bytes.
+
+Like in qcow2, directly after the image header, optional sections called 
header extensions can
+be stored. Each extension has a structure like the following:
+
+    Byte  0 -  3:   Header extension type:
+                        0x00000000 - End of the header extension area
+                        other      - Unknown header extension, can be safely
+                                     ignored
+
+          4 -  7:   Length of the header extension data
+
+          8 -  n:   Header extension data
+
+          n -  m:   Padding to round up the header extension size to the next
+                    multiple of 8.
+
+Unless stated otherwise, each header extension type shall appear at most once
+in the same image.
+
+== Cluster mapping ==
+
+QDB uses a ONE-level structure for the mapping of
+bitmaps to host clusters. It is called L1 table.
+
+The L1 table has a variable size (stored in the Bitmap table entry) and may
+use multiple clusters, however it must be contiguous in the QDB file.
+
+Given a offset into the bitmap, the offset into the QDB file can be
+obtained as follows:
+
+    offset = l1_table[offset / cluster_size] + (offset % cluster_size)
+
+L1 table entry:
+
+    Bit  0 -  61:   Cluster descriptor
+
+        62 -  63:   Reserved
+
+Standard Cluster Descriptor (the same as in qcow2):
+
+    Bit       0:    If set to 1, the cluster reads as all zeros. The host
+                    cluster offset can be used to describe a preallocation,
+                    but it won't be used for reading data from this cluster,
+                    nor is data read from the backing file if the cluster is
+                    unallocated.
+
+         1 -  8:    Reserved (set to 0)
+
+         9 - 55:    Bits 9-55 of host cluster offset. Must be aligned to a
+                    cluster boundary. If the offset is 0, the cluster is
+                    unallocated.
+
+        56 - 61:    Reserved (set to 0)
+
+If a cluster is unallocated, read requests shall read zero.
+
+== Bitmap table ==
+
+QDB supports storing of several bitmaps.
+
+A directory of all bitmaps is stored in the bitmap table, a contiguous area
+in the QDB file, whose starting offset and length are given by the header
+fields bitmaps_offset and nb_bitmaps. The entries of the bitmap table
+have variable length, depending on the length of name and extra data.
+
+Bitmap table entry:
+
+    Byte 0 -  7:    Offset into the QDB file at which the L1 table for the
+                    bitmap starts. Must be aligned to a cluster boundary.
+
+         8 - 11:    Number of entries in the L1 table of the bitmap
+
+        12 - 15:    Bitmap granularity
+                    As represented in HBitmap structure. Given a granularity of
+                    G, each bit in the bitmap will actually represent a group
+                    of 2^G bytes.
+
+        16 - 23:    Bitmap size
+                    The size of really stored data is
+                    (size + (1 << granularity) - 1) >> granularity
+
+             24:    Bitmap enabled flag (valid values are 1 and 0)
+
+             25:    File dirty flag (valid values are 1 and 0;
+                    if value is 1 the bitmap is inconsistent)
+
+        26 - 27:    Size of the bitmap name
+
+        36 - 39:    Size of extra data in the table entry (used for future
+                    extensions of the format)
+
+        variable:   Extra data for future extensions. Unknown fields must be
+                    ignored.
+
+        variable:   Name of the bitmap (not null terminated)
+
+        variable:   Padding to round up the bitmap table entry size to the
+                    next multiple of 8.
+
+The fields "size", "granularity", "enabled" and "name" are corresponding with
+the fields in struct BdrvDirtyBitmap.
-- 
1.9.1