Re: [PATCH v3 00/14] File-based migration support and fixed-ram features

[Daniel P . Berrangé]

On Mon, Mar 20, 2023 at 12:14:53PM +0100, Claudio Fontana wrote:
> (adding Fabiano to the thread)
> 
> On 2/10/23 16:35, Daniel P. Berrangé wrote:
> > On Thu, Feb 09, 2023 at 02:32:01PM +0100, Claudio Fontana wrote:
> >> Hello Daniel and all,
> >>
> >> resurrecting this series from end of last year,
> >>
> >> do we think that this is the right approach and first step to
> >> be able to provide good performance for virsh save and virsh
> >> restore?
> > 
> > I've looked through the series in some more detail now and will
> > send review comments separately. Overall, I'm pretty pleased with
> > the series and think it is on the right path. The new format it
> > provides should be amenable to parallel I/O with multifd and
> > be able to support O_DIRECT to avoid burning through the host I/O
> > cache.
> 
> Just wanted to add a thought we had with Fabiano a few days ago:
> 
> experimentally, it is clear that fixed-ram is an optimization, but the actual 
> scalability seems to come from the successive parallel I/O with multifd.
> 
> Since the goal is being able to transfer _to disk_ (fdatasync) the whole 30G 
> memory in 5 seconds, the need to split the cpu-intensive work into smaller 
> tasks remains,
> and the main scalability solution seems to come from the multifd part of the 
> work (or another way to split the problem), combined with the O_DIRECT 
> friendliness to avoid the trap of the cache trashing.
> 
> Not adding much, just highlighting that fixed-ram _alone_ does not seem to 
> suffice, we probably need all pieces of the puzzle in place.

Agreed, the only thing that fixed-ram alone really does is to
ensure you have a finite file size if you're saving the state
while the VM is running, because successive dirtying of the
same block will write to the same location in file. Nice to
avoid wasting storage and gives a small speed up on restore.

It isn't going to give the massive speedup you're looking for
though. multifd + O_DIRECT is a critical followup piece to
achieve the major performance improvement. The fixed-ram
feature makes it practical to use multifd, without needing
to split content streams across many files, or have to invent
a way to splice multiple streams into the same file.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|