Re: [Qemu-devel] [PATCH] qemu-kvm: introduce cpu_start/cpu_stop commands

[Anthony Liguori]

On 11/23/2010 08:00 AM, Avi Kivity wrote:
> > > If we could catch SIGSTOP, then it would be easy to unblock it only 
> > > while running in guest context. It would then stop on exit to 
> > > userspace.
> >
> > Yeah, that's not a bad idea.
>
> Except we can't.

Yeah, I s:SIGSTOP:SIGUSR1:g.

> > > Using monitor commands is fairly heavyweight for something as high 
> > > frequency as this.  What control period do you see people using?  
> > > Maybe we should define USR1 for vcpu start/stop.
> > >
> > > What happens if one vcpu is stopped while another is running?  Spin 
> > > loops, synchronous IPIs will take forever.  Maybe we need to stop 
> > > the entire process.
> >
> > It's the same problem if a VCPU is descheduled while another is running. 
>
> We can fix that with directed yield or lock holder preemption 
> prevention.  But if a vcpu is stopped by qemu, we suddenly can't.

That only works for spin locks.

Here's the scenario:

1) VCPU 0 drops to userspace and acquires qemu_mutex
2) VCPU 0 gets descheduled
3) VCPU 1 needs to drop to userspace and acquire qemu_mutex, gets 
blocked and yields
4) If we're lucky, VCPU 0 gets scheduled but it depends on how busy the 
system is

With CFS hard limits, once (2) happens, we're boned for (3) because (4) 
cannot happen.  By having QEMU know about (2), it can choose to run just 
a little bit longer in order to drop qemu_mutex such that (3) never happens.

> > The problem with stopping the entire process is that a big motivation 
> > for this is to ensure that benchmarks have consistent results 
> > regardless of CPU capacity.  If you just monitor the full process, 
> > then one VCPU may dominate the entitlement resulting in very erratic 
> > benchmarking.
>
> What's the desired behaviour?  Give each vcpu 300M cycles per second, 
> or give a 2vcpu guest 600M cycles per second?

Each vcpu gets 300M cycles per second.

> You could monitor threads separately but stop the entire process.  
> Stopping individual threads will break apart as soon as they start 
> taking locks.

I don't think so..  PLE should work as expected.  It's no different than 
a normally contended system.

Regards,

Anthony Liguori