[Qemu-block] RFC: Status of the intermediate block streaming work

[Alberto Garcia]

Hi all,

I have been retaking my work on the intermediate block streaming
patches. Here's a summary of this feature and the current problems:

The goal is to be able to perform a 'block-stream' operation on a node
different from the active layer:

[A] <- [B] <- [C] <- [D] <- [E]

In this case, we would be able to copy the data from [C] into [D],
which would result in the following graph:

[A] <- [B] <- [D] <- [E]

The block job would be created in [D], and all nodes that are not
involved in the operation would be free for other block jobs. That
would allow us to run several block-stream commands in parallel, and
in the example above we would be able to copy [A] into [B] and [C]
into [D] in parallel, which would result in this chain:

[B] <- [D] <- [E]

The code has been ready for more than half a year, but there are
currently two problems that prevent it from working:

  1) The destination images need to be reopened in read-write
     mode. This has the side effect of reopening its whole backing
     chain in read-only mode. If there's a block job writing in one of
     those images it will fail.

     In the example above, if we stream [A] into [B] and then try to
     stream [C] into [D], then [B] will switch to read-only mode and
     the first operation will fail.

     The problem is described here:

     https://lists.gnu.org/archive/html/qemu-block/2015-10/msg01042.html

     As I understand it, Kevin's work on bdrv_reopen() will fix that.

  2) When a block-stream operation calls bdrv_reopen(), it drains all
     images. This can trigger the completion of a different
     block-stream operation, that will remove the intermediate images
     from the chain. When the original bdrv_reopen() continues, it
     still contains pointers to the images that have just been
     removed, crashing QEMU.

     The problem is described here:

     https://lists.gnu.org/archive/html/qemu-block/2015-06/msg00701.html

     I'm still not sure about how to deal with this. I've been taking
     a look at the bdrv_drained_begin/end() API, but as I understand
     it it prevents requests from a different AioContext.  Since all
     BDS in the same chain share the same context it does not really
     help here.

Since the ability to run two jobs in parallel is of secondary
importance to me, I'm considering to disallow it by locking the whole
chain until these two issues have been solved, but any suggestions are
more than welcome.

Thanks,

Berto