[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication
From: |
Benoît Canet |
Subject: |
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication |
Date: |
Thu, 3 Jan 2013 18:18:40 +0100 |
User-agent: |
Mutt/1.5.21 (2010-09-15) |
Hello,
I started to write the deduplication metrics code in order to be able
to design asynchronous deduplication.
I am looking for a way to create a metric allowing deduplication to be paused
or resumed on a given threshold.
Does anyone have a sugestion regarding the metric that could be used for this ?
Best regards
Benoît
> Le Wednesday 02 Jan 2013 à 17:16:03 (+0100), Benoît Canet a écrit :
> This patchset is a cleanup of the previous QCOW2 deduplication rfc.
>
> One can compile and install https://github.com/wernerd/Skein3Fish and use the
> --enable-skein-dedup configure option in order to use the faster skein HASH.
>
> Images must be created with "-o dedup=[skein|sha256]" in order to activate the
> deduplication in the image.
>
> Deduplication is now fast enough to be usable.
>
> v4: Fix and complete qcow2 spec [Stefan]
> Hash the hash_algo field in the header extension [Stefan]
> Fix qcow2 spec [Eric]
> Remove pointer to hash and simplify hash memory management [Stefan]
> Rename and move qcow2_read_cluster_data to qcow2.c [Stefan]
> Document lock dropping behaviour of the previous function [Stefan]
> cleanup qcow2_dedup_read_missing_cluster_data [Stefan]
> rename *_offset to *_sect [Stefan]
> add a ./configure check for ssl [Stefan]
> Replace openssl by gnutls [Stefan]
> Implement Skein hashes
> Rewrite pretty every qcow2-dedup.c commits after Add
> qcow2_dedup_read_missing_and_concatenate to simplify the code
> Use 64KB deduplication hash block to reduce allocation flushes
> Use 64KB l2 tables to reduce allocation flushes [breaks compatibility]
> Use lazy refcounts to avoid qcow2_cache_set_dependency loops resultings
> in frequent caches flushes
> Do not create and load dedup RAM structures when bdrs->read_only is true
>
> v3: make it work barely
> replace kernel red black trees by gtree.
>
> *** BLURB HERE ***
>
> Benoît Canet (30):
> qcow2: Add deduplication to the qcow2 specification.
> qcow2: Add deduplication structures and fields.
> qcow2: Add qcow2_dedup_read_missing_and_concatenate
> qcow2: Make update_refcount public.
> qcow2: Create a way to link to l2 tables when deduplicating.
> qcow2: Add qcow2_dedup and related functions
> qcow2: Add qcow2_dedup_store_new_hashes.
> qcow2: Implement qcow2_compute_cluster_hash.
> qcow2: Extract qcow2_dedup_grow_table
> qcow2: Add qcow2_dedup_grow_table and use it.
> qcow2: create function to load deduplication hashes at startup.
> qcow2: Load and save deduplication table header extension.
> qcow2: Extract qcow2_do_table_init.
> qcow2-cache: Allow to choose table size at creation.
> qcow2: Add qcow2_dedup_init and qcow2_dedup_close.
> qcow2: Extract qcow2_add_feature and qcow2_remove_feature.
> block: Add qemu-img dedup create option.
> qcow2: Behave correctly when refcount reach 0 or 2^16.
> qcow2: Integrate deduplication in qcow2_co_writev loop.
> qcow2: Serialize write requests when deduplication is activated.
> qcow2: Add verification of dedup table.
> qcow2: Adapt checking of QCOW_OFLAG_COPIED for dedup.
> qcow2: Add check_dedup_l2 in order to check l2 of dedup table.
> qcow2: Do not overwrite existing entries with QCOW_OFLAG_COPIED.
> qcow2: Integrate SKEIN hash algorithm in deduplication.
> qcow2: Add lazy refcounts to deduplication to prevent
> qcow2_cache_set_dependency loops
> qcow2: Use large L2 table for deduplication.
> qcow: Set dedup cluster block size to 64KB.
> qcow2: init and cleanup deduplication.
> qemu-iotests: Filter dedup=on/off so existing tests don't break.
>
> block/Makefile.objs | 1 +
> block/qcow2-cache.c | 12 +-
> block/qcow2-cluster.c | 116 +++--
> block/qcow2-dedup.c | 1157
> ++++++++++++++++++++++++++++++++++++++++++
> block/qcow2-refcount.c | 157 ++++--
> block/qcow2.c | 357 +++++++++++--
> block/qcow2.h | 120 ++++-
> configure | 55 ++
> docs/specs/qcow2.txt | 100 +++-
> include/block/block_int.h | 1 +
> tests/qemu-iotests/common.rc | 3 +-
> 11 files changed, 1955 insertions(+), 124 deletions(-)
> create mode 100644 block/qcow2-dedup.c
>
> --
> 1.7.10.4
>
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, (continued)
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Troy Benjegerdes, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Benoît Canet, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, ronnie sahlberg, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Benoît Canet, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Troy Benjegerdes, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, ronnie sahlberg, 2013/01/02
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Stefan Hajnoczi, 2013/01/03
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Troy Benjegerdes, 2013/01/03
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Dietmar Maurer, 2013/01/04
- Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication, Stefan Hajnoczi, 2013/01/04
Re: [Qemu-devel] [RFC V4 00/30] QCOW2 deduplication,
Benoît Canet <=