RE: sysemu SMP scheduling

[Brian Cain]

> -----Original Message-----
> From: Brian Cain
> Sent: Wednesday, April 28, 2021 10:06 PM
> To: qemu-devel@nongnu.org
> Cc: Richard Henderson <richard.henderson@linaro.org>; Taylor Simpson
> <tsimpson@quicinc.com>; Michael Lambert <mlambert@quicinc.com>;
> Manning, Sid <sidneym@quicinc.com>
> Subject: sysemu SMP scheduling
> 
> For some hexagon use cases, we would prefer to have finer grained scheduling
> among multiple guest cores/threads.  We haven't been able to determine
> exactly what kind of scheduling algorithm is operating in the baseline case.  
> If
> the current hw thread is ready-to-run and is spinning over a tight loop that
> never hits any exceptions, would it ever yield to another thread after so-many
> iterations/TBs executed?  Or perhaps since we're executing translated blocks
> there's just no yield opportunity available?
> 
> We came up with a design for this finer-grained scheduling feature, but are 
> re-
> examining whether or not it should be necessary and if it is necessary, 
> whether
> it should have been designed like so.  We haven't seen a similar example in
> other targets, so we'd love to get feedback on the approach.
> 
> In the TranslatorOps .tb_stop() we generate code like so:
> 
>       if the current count of ready-to-run threads >= 2:
>               tb_count++
>               if tb_count > THRESHOLD:
>                       gen_exception(EXCP_YIELD);
>                       tb_count = 0
>                       gen exit_tb

Hmm, I omitted some critical details of this algorithm.

tb_stop, corrected:

        if the current count of ready-to-run threads >= 2:
                consecutive_tb_count++
                if consecutive_tb_count > THRESHOLD:
                        gen_exception(EXCP_YIELD);
                        consecutive_tb_count = 0
                        gen exit_tb
        last_cpu_id = this_cpu_id

tb_start generates code like this:

        if this_cpu_id != last_cpu_id:
                consecutive_tb_count = 0

> - "current count of ready-to-run threads" is based on the values in the CPU
> state.  When entering a wait/halt mode, we set the appropriate state and call
> cpu_stop_current().
> - Is EXCP_YIELD an appropriate mechanism for this feature?  It seems like
> maybe it has no special handling, but any exception can trigger a yield?
> 
> -Brian