[Qemu-devel] [PATCH v10 3/9] Add XBZRLE documentation

[Orit Wasserman]

Signed-off-by: Orit Wasserman <address@hidden>
---
 docs/xbzrle.txt |   97 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 97 insertions(+), 0 deletions(-)
 create mode 100644 docs/xbzrle.txt

diff --git a/docs/xbzrle.txt b/docs/xbzrle.txt
new file mode 100644
index 0000000..aafdb84
--- /dev/null
+++ b/docs/xbzrle.txt
@@ -0,0 +1,97 @@
+XBZRLE (Xor Based Zero Run Length Encoding)
+===========================================
+
+Using XBZRLE (Xor Based Zero Run Length Encoding) allows for the reduction of 
VM downtime
+and the total live-migration time of Virtual machines. It is particularly 
useful for virtual machines running memory write intensive workloads that are 
typical of large enterprise applications such as SAP ERP Systems, and generally
+speaking for any application that uses a sparse memory update pattern.
+
+Instead of sending the changed guest memory page this solution will send a 
compressed version of the updates, thus reducing the amount of data sent during 
live migration.
+In order to be able to calculate the update, the previous memory pages needed 
to be stored. Those pages are stored in a dedicated cache (hash table) and are 
accessed by their address.
+The larger the cache size the better the chances are that the page has already 
been stored in the cache. A Small cache size will result in high cache miss 
rate.
+
+Format
+=======
+
+The compression format uses the zero value, where zero represents an unchanged 
value.
+The page data delta is represented by zero and non zero runs.
+A zero run is represented by it's length (in bytes).
+A non zero run is represented by it's length (in bytes) and the data.
+The run length is encoded using ULEB128 (http://en.wikipedia.org/wiki/LEB128)
+
+page = zrun nzrun
+       | zrun nzrun page
+
+zrun = length
+
+nzrun = length byte...
+
+length = uleb128 encoded integer
+
+On the sender side XBZRLE is used as a compact delta encoding of page updates,
+retrieving the old page content from the cache (default size of 512 MB). The
+receiving side uses the existing page's content and XBZRLE to decode the new 
page's content.
+
+This is a more compact way to store the deltas than the previous version.
+
+This work was originally based on research results published VEE 2011: 
Evaluation of
+Delta Compression Techniques for Efficient Live Migration of Large Virtual
+Machines by Benoit, Svard, Tordsson and Elmroth. Additionally the delta encoder
+XBRLE was improved further using the XBZRLE instead.
+
+XBZRLE has a sustained bandwidth of 2-2.5 GB/s for typical workloads making it
+ideal for in-line, real-time encoding such as is needed for live-migration.
+
+Migration Capabilities
+======================
+In order to use XBZRLE the destination QEMU version should be able to
+decode the new format.
+Adding a new migration capabilities command that will allow external management
+to query for it support.
+A typical use for the destination
+    {qemu} info migrate_capabilities
+    {qemu} xbzrle, ...
+
+In order to enable capabilities for future live migration,
+a new command migrate_set_parameter is introduced:
+    {qemu} migrate_set_parameter xbzrle
+
+Usage
+======
+
+1. Activate xbzrle
+2. Set the XBZRLE cache size - the cache size is in MBytes and should be a 
power of 2. The cache default value is 64MBytes.
+3. start outgoing migration
+
+A typical usage scenario:
+    {qemu} migrate_set_parameter xbzrle
+    {qemu} migrate_set_cachesize 256m
+    {qemu} migrate -d tcp:destination.host:4444
+    {qemu} info migrate
+    ...
+    transferred ram-duplicate: A kbytes
+    transferred ram-normal: B kbytes
+    transferred ram-xbrle: C kbytes
+    overflow ram-xbrle: D pages
+    cache-miss ram-xbrle: E pages
+
+cache-miss: the number of cache misses to date - high cache-miss rate
+indicates that the cache size is set too low.
+overflow: the number of overflows in the decoding which where the delta could 
not be compressed. This can happen if the changes in the pages are too large
+or there are many short changes for example change every second byte (half a 
page).
+
+Testing: Testing indicated that live migration with XBZRLE was completed in 
110 seconds, whereas without it would not be able to complete.
+
+A simple synthetic memory r/w load generator:
+..    include <stdlib.h>
+..    include <stdio.h>
+..    int main()
+..    {
+..        char *buf = (char *) calloc(4096, 4096);
+..        while (1) {
+..            int i;
+..            for (i = 0; i < 4096 * 4; i++) {
+..                buf[i * 4096 / 4]++;
+..            }
+..            printf(".");
+..        }
+..    }
-- 
1.7.7.6