Re: [Qemu-arm] [Qemu-devel] [PATCH 11/13] target/arm/cpu64: max cpu: Introduce sve-vls-map

[Andrew Jones]

On Mon, May 13, 2019 at 01:41:26PM +0100, Dave Martin wrote:
> On Mon, May 13, 2019 at 01:30:23PM +0100, Andrew Jones wrote:
> > On Mon, May 13, 2019 at 12:26:36PM +0100, Dave Martin wrote:
> > > On Sun, May 12, 2019 at 09:36:22AM +0100, Andrew Jones wrote:
> > > > Introduce another cpu property to control SVE vector lengths,
> > > > sve-vls-map, which allows the user to explicitly select the
> > > > set of vector lengths the guest can use. The map must conform
> > > > to QEMU's limits and architectural constraints, checked when
> > > > the property is set. Inconsistencies with sve-max-vq are also
> > > > checked. The bit number of a set bit in the map represents the
> > > > allowed vector length in number of quadwords.
> > > > 
> > > > Note, as the map is implemented with a single 64-bit word we
> > > > currently only support up to 8192-bit vectors. As QEMU and
> > > > KVM only support up to 2048-bit vectors then this sufficient
> > > > now, and probably for some time. Extending the bitmap beyond
> > > > a single word will likely require changing the property to
> > > > a string and adding yet another parser to QEMU.
> > > > 
> > > > Signed-off-by: Andrew Jones <address@hidden>
> > > > ---
> > > >  target/arm/cpu.c     |  4 +++
> > > >  target/arm/cpu.h     |  3 ++
> > > >  target/arm/cpu64.c   | 70 +++++++++++++++++++++++++++++++++++++++++---
> > > >  target/arm/helper.c  | 10 ++++++-
> > > >  target/arm/monitor.c |  9 ++++--
> > > >  5 files changed, 88 insertions(+), 8 deletions(-)
> > > > 
> > > 
> > > [...]
> > > 
> > > > diff --git a/target/arm/cpu.h b/target/arm/cpu.h
> > > > index 8292d547e8f9..f0d0ce759ba8 100644
> > > > --- a/target/arm/cpu.h
> > > > +++ b/target/arm/cpu.h
> > > > @@ -920,6 +920,9 @@ struct ARMCPU {
> > > >  
> > > >      /* Used to set the maximum vector length the cpu will support.  */
> > > >      uint32_t sve_max_vq;
> > > > +
> > > > +    /* Each bit represents a supported vector length of (bitnum * 16) 
> > > > bytes */
> > > > +    uint64_t sve_vls_map;
> > > 
> > > Just to be clear, the representation here is different from the
> > > representation in KVM_REG_ARM64_SVE_VLS?
> > > 
> > > In the latter, bit n represents vector length ((n + 1) * 16) bytes.
> > 
> > KVM also uses bitnum * 16. bitnum is the the bit number, not the shift.
> 
> Can you point to the relevant kernel code?  This isn't what I thought I
> wrote...

The Documentation/virtual/kvm/api.txt documentation has

 > if (vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX &&
 >    ((vector_lengths[(vq - KVM_ARM64_SVE_VQ_MIN) / 64] >>
 >                ((vq - KVM_ARM64_SVE_VQ_MIN) % 64)) & 1))
 >        /* Vector length vq * 16 bytes supported */
 > else
 >        /* Vector length vq * 16 bytes not supported */

Taking vq=1, we check (vector_lengths[0] >> 0) to see if we have the
length (1 * 16) bytes. Since bitnum 1 is (1 << 0) that means the shift
is zero and (bitnum * 16) is the same as (vq * 16). With ((n + 1) * 16),
n would have to be zero, which is not a valid bitnum, but is a valid
bit shift.

> 
> > So you can't have bitnum=0. 'n' must be a shift, as it would need to
> > allow zero in order to represent KVM_ARM64_SVE_VQ_MIN.
> 
> [...]
> 
> > > > +static void cpu_set_sve_vls_map(Object *obj, Visitor *v, const char 
> > > > *name,
> > > > +                                void *opaque, Error **errp)
> > > > +{
> > > > +    ARMCPU *cpu = ARM_CPU(obj);
> > > > +    Error *err = NULL;
> > > > +    uint64_t mask = ~(BIT_MASK(ARM_MAX_VQ - 1) - 1);
> > > > +    int i;
> > > > +
> > > > +    visit_type_uint64(v, name, &cpu->sve_vls_map, errp);
> > > > +
> > > > +    if (!err) {
> > > > +        if (cpu->sve_vls_map == 0) {
> > > > +            error_setg(&err, "SVE vector length map cannot be zero");
> > > 
> > > Maybe say "empty" here, since the map represents a set?
> > 
> > Empty does sound better when considering the map represents a set, but
> > the user will be inputting a number for the map, so zero will be what
> > they attempted to input.
> 
> If the user is inputting the encoded number directly, I guess it makes
> sense to describe the empty set as "zero".
> 
> > > (But it this is just for debug rather than reporting errors to the user,
> > > it probably doesn't matter much.)
> > 
> > I'm also not too worried about which term we use, so I'm happy to change
> > it if you'd like.
> 
> I guess I don't have a strong opinion.
> 
> [...]
> 
> > > > diff --git a/target/arm/helper.c b/target/arm/helper.c
> > > > index 1e6eb0d0f360..bedec1ea0b27 100644
> > > > --- a/target/arm/helper.c
> > > > +++ b/target/arm/helper.c
> > > > @@ -5254,12 +5254,20 @@ uint32_t sve_zcr_len_for_el(CPUARMState *env, 
> > > > int el)
> > > >  static void zcr_write(CPUARMState *env, const ARMCPRegInfo *ri,
> > > >                        uint64_t value)
> > > >  {
> > > > +    ARMCPU *cpu = arm_env_get_cpu(env);
> > > >      int cur_el = arm_current_el(env);
> > > >      int old_len = sve_zcr_len_for_el(env, cur_el);
> > > >      int new_len;
> > > >  
> > > >      /* Bits other than [3:0] are RAZ/WI.  */
> > > > -    raw_write(env, ri, value & 0xf);
> > > > +    value &= 0xf;
> > > 
> > > You might want to sanity-check that the max vq you configured for the
> > > vcpu is <= 16 here.
> > 
> > It's already checked by the cpu property set function at input time.
> 
> The check is in one place, but the assumptions based on it are
> potentially all over the place -- it may be tricky to track them
> all down if you ever add support for >8192-bit vectors.
> 
> (Not a big deal from my point of view, but I'd migration to larger
> vectors to be as smooth as reasonably possible if we ever find we have
> to do it.)
>

Adding asserts might be a good idea to catch bugs when expanding the
supported vector lengths. I can certainly add one here. Maybe Richard
has some ideas as well for this.

Thanks,
drew