Re: [Qemu-block] [Qemu-devel] Can I only commit from active image to corresponding range of its backing file by qemu cmd?

[lampahome]

*(That's what I said range is different)*
Ex: 1st chunk of device will save into image.000
2nd chunk of device will save into image.001
Nth chunk of device will save into image.(N-1)
...etc

I can see all block device data when I mount image.(N-1) by qemu-nbd cuz
the chunk doesn't overlap and all chunks connect by backing chain.

How exactly did you create those images?  I'm trying to verify the steps you used to split the image. I know the concept of the split, but without seeing actual commands used, I don't know that you actually accomplished the split in the manner desired.  (It's okay if a reproduction uses smaller scales for speed, such as splitting a 32M image across 1M qcow2 files - the point remains that seeing the actual steps used may offer additional insights into your usage scenario).

I create those images by my own python scriopt. I research the behaviors when read/write/discard...etc from qemu github.

If every image size limit I set is 1TB, script will create next image automatically when refcount of the whole image is all up to 1 and then redirect read/write to the new image.

  

Or are you trying to ask if it is possible to create such a fragmented design with current tools?  (The answer that we've given you is that no, it is not easy to do, because no one has needed it so far).  There's no way to tell a running qemu that writes to offsets 0-1M go into one file, while writes to offsets 1M to 2M go into another - ALL writes go into the currently active layer, regardless of the offset represented by the write.

Got it, thx.

 

So now I have two *version of block device(like concept of snapshot)*:
One is image.000 to image.(N-1). I can access the data before modify by
mount image.(N-1) through qemu-nbd
The other one is image.000 to image.N.  I can access the data after modify
by mount image.N through qemu-nbd(cuz the visible 1st chunk are in the
image.N)

Consider about the situation:
000   A - - - - - - - -  <<<<<---  store the 1st chunk of block device
001   - B - - - - - - -
002   - - C - - - - - - (1st state of block device)
003   A' - - - - - - - - <<<<<--- store the 1st chunk of block device, but
data is different
004   - - - D - - - - - (2nd state of block device)
005   - - - - E - - - -  (3rd state of block device)

The original problem is If I want to remove the 2nd state(003 and 004) but
I need to keep the data of 003 and 004.
If I just commit 003, the A' of 003 must be committed into 002 cuz 002 is
the backing file of 003.
I try to figure out some way to let it only commit from 003 into 000.

I'm not quite following your diagram. My naive read (probably wrong) is that you are trying to present a 9M image (scaled M to G or T as appropriate) to the guest, as represented by the 9 characters, but that the initial image only populated 3M of the 9 with guest-visible contents represented by ABC------.  So you want to split that into files 000 containing offsets 0-1M (A--------), 001 containing offsets 1M-2M (-B-------), and 002 containing offsets 2M-3M (--C------).  Then you want to run the guest, which does some modifications in offsets 0-1M (I'll write it as "a" instead of "A'", you could also have chosen a different letter except that your example already uses "D" elsewhere), so the guest now sees (aBC------), and you want to store that incremental backup in file 003, containing just (a--------).  But that's where I got confused - my original assumption was that 003 represented offsets 3M-4M (---X-----), but you are now showing it as representing offsets 0-1M.  It's also not clear which files in your list have which other files as backing files.

You can view it as snapshots. From 000~002 are the data of 1st snapshot. and 003~004 are the 2nd snapshot. The difference of two snapshot is the 1st chunk(A is in the 1st snapshot and A' is in the 2nd snapshot).

So that's why the A' writes in 1st chunk of 003(because 1st of 003 and 000 are the different data).

Incremental backup by programming way is just a thought because I try to conquer the file size limit

I indeed try rebase and commit command to do this, but still can't do this.

Now maybe the case is not normal by everyone. I try to figure out another way or use case to try incremental backup by my own script. 

 

--
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3266
Virtualization:  qemu.org | libvirt.org