[Qemu-devel] Re: [PATCH 02/10] Add buffered_file_internal constant

[Anthony Liguori]

On 11/30/2010 12:04 PM, Juan Quintela wrote:
> Anthony Liguori<address@hidden>  wrote:
>    
> > On 11/30/2010 10:32 AM, Juan Quintela wrote:
> >      
> > > "Michael S. Tsirkin"<address@hidden>   wrote:
> > >
> > >        
> > > > On Tue, Nov 30, 2010 at 04:40:41PM +0100, Juan Quintela wrote:
> > > >
> > > >          
> > > > > Basically our bitmap handling code is "exponential" on memory size,
> > > > >
> > > > >            
> > > > I didn't realize this. What makes it exponential?
> > > >
> > > >          
> > > Well, 1st of all, it is "exponential" as you measure it.
> > >
> > > stalls by default are:
> > >
> > > 1-2GB: milliseconds
> > > 2-4GB: 100-200ms
> > > 4-8GB: 1s
> > > 64GB: 59s
> > > 400GB: 24m (yes, minutes)
> > >
> > > That sounds really exponential.
> > >
> > >        
> > How are you measuring stalls btw?
> >      
> At the end of the ram_save_live().  This was the reason that I put the
> information there.
>
> for the 24mins stall (I don't have that machine anymore) I had less
> "exact" measurements.  It was the amount that it "decided" to sent in
> the last non-live part of memory migration.  With the stalls&  zero page
> account, we just got to the point where we had basically infinity speed.
>    

That's not quite guest visible.

It only is a "stall" if the guest is trying to access device emulation 
and acquiring the qemu_mutex.  A more accurate measurement would be 
something that measured guest availability.  For instance, I tight loop 
of while (1) { usleep(100); gettimeofday(); } that then recorded periods 
of unavailability > X.

Of course, it's critically important that a working version of pvclock 
be available int he guest for this to be accurate.

Regards,

Anthony Liguori

> Later, Juan.
>
>    
> > Regards,
> >
> > Anthony Liguori
> >
> >      
> > > Now the other thing is the cache size.
> > >
> > > with 64GB of RAM, we basically have a 16MB bitmap size, i.e. we blow the
> > > cache each time that we run ram_save_live().
> > >
> > > This is one of the reasons why I don't want to walk the bitmap on
> > > ram_save_remaining().
> > >
> > > ram_save_remaining() is linear with memory size (previous my changes).
> > > ram_save_live is also linear on the memory size, so we are in a worse
> > > case of n^n  (notice that this is the worst case, we normally do much
> > > better n^2, n^3 or so).
> > >
> > > Add to this that we are blowing the caches with big amounts of memory
> > > (we don't do witch smaller sizes), and you can see that our behaviour is
> > > clearly exponential.
> > >
> > > As I need to fix them, I will work on them today/tomorrow.
> > >
> > > Later, Juan.
> > >
> > >