Re: [PATCH v7 3/9] i386: hardcode supported eVMCS version to '1'

[Eduardo Habkost]

On Fri, Jun 04, 2021 at 09:28:15AM +0200, Vitaly Kuznetsov wrote:
> Eduardo Habkost <ehabkost@redhat.com> writes:
> 
> > On Thu, Jun 03, 2021 at 01:48:29PM +0200, Vitaly Kuznetsov wrote:
> >> Currently, the only eVMCS version, supported by KVM (and described in TLFS)
> >> is '1'. When Enlightened VMCS feature is enabled, QEMU takes the supported
> >> eVMCS version range (from KVM_CAP_HYPERV_ENLIGHTENED_VMCS enablement) and
> >> puts it to guest visible CPUIDs. When (and if) eVMCS ver.2 appears a
> >> problem on migration is expected: it doesn't seem to be possible to migrate
> >> from a host supporting eVMCS ver.2 to a host, which only support eVMCS
> >> ver.1.
> >
> > Isn't it possible and safe to expose eVMCS ver.1 to the guest on
> > a host that supports ver.2?
> 
> We expose the supported range, guest is free to use any eVMCS version in
> the range (see below):

Oh, I didn't notice the returned value was a range.

> 
> >
> >> 
> >> Hardcode eVMCS ver.1 as the result of 'hv-evmcs' enablement for now. Newer
> >> eVMCS versions will have to have their own enablement options (e.g.
> >> 'hv-evmcs=2').
> >> 
> >> Signed-off-by: Vitaly Kuznetsov <vkuznets@redhat.com>
> >> ---
> >>  docs/hyperv.txt       |  2 +-
> >>  target/i386/kvm/kvm.c | 16 +++++++++++-----
> >>  2 files changed, 12 insertions(+), 6 deletions(-)
> >> 
> >> diff --git a/docs/hyperv.txt b/docs/hyperv.txt
> >> index a51953daa833..000638a2fd38 100644
> >> --- a/docs/hyperv.txt
> >> +++ b/docs/hyperv.txt
> >> @@ -170,7 +170,7 @@ Recommended: hv-frequencies
> >>  3.16. hv-evmcs
> >>  ===============
> >>  The enlightenment is nested specific, it targets Hyper-V on KVM guests. 
> >> When
> >> -enabled, it provides Enlightened VMCS feature to the guest. The feature
> >> +enabled, it provides Enlightened VMCS version 1 feature to the guest. The 
> >> feature
> >>  implements paravirtualized protocol between L0 (KVM) and L1 (Hyper-V)
> >>  hypervisors making L2 exits to the hypervisor faster. The feature is 
> >> Intel-only.
> >>  Note: some virtualization features (e.g. Posted Interrupts) are disabled 
> >> when
> >> diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
> >> index c676ee8b38a7..d57eede5dc81 100644
> >> --- a/target/i386/kvm/kvm.c
> >> +++ b/target/i386/kvm/kvm.c
> >> @@ -1490,13 +1490,19 @@ static int hyperv_init_vcpu(X86CPU *cpu)
> >>          ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
> >>                                    (uintptr_t)&evmcs_version);
> >>  
> >> -        if (ret < 0) {
> >> -            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
> >> -                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
> >> +        /*
> >> +         * KVM is required to support EVMCS ver.1. as that's what 
> >> 'hv-evmcs'
> >> +         * option sets. Note: we hardcode the maximum supported eVMCS 
> >> version
> >> +         * to '1' as well so 'hv-evmcs' feature is migratable even when 
> >> (and if)
> >> +         * ver.2 is implemented. A new option (e.g. 'hv-evmcs=2') will 
> >> then have
> >> +         * to be added.
> >> +         */
> >> +        if (ret < 0 || (uint8_t)evmcs_version > 1) {
> >
> > Wait, do you really want to get a fatal error every time, after a
> > kernel upgrade?
> >
> 
> Here, evmcs_version (returned by kvm_vcpu_enable_cap()) represents a
> *range* of supported eVMCS versions:
> 
> (evmcs_highest_supported_version << 8) | evmcs_lowest_supported_version
> 
> Currently, this is 0x101 [1..1] range.
> 
> The '(uint8_t)evmcs_version > 1' check here means 'eVMCS v1' is no
> longer supported by KVM. This is not going to happen any time soon, but
> I can imagine in 10 years or so we'll be dropping v1 so the range (in
> theory) can be [10..2] -- which would mean eVMCS ver. 1 is NOT
> supported. And we can't proceed then.

Where is this documented?  The only reference to
KVM_CAP_HYPERV_ENLIGHTENED_VMCS I've found in linux/Documentation is this
single sentence:

| - HYPERV_CPUID_NESTED_FEATURES leaf and HV_X64_ENLIGHTENED_VMCS_RECOMMENDED
|  feature bit are only exposed when Enlightened VMCS was previously enabled
|  on the corresponding vCPU (KVM_CAP_HYPERV_ENLIGHTENED_VMCS).


> >>          ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
> >>                                    (uintptr_t)&evmcs_version);
> >>  
> >> -        if (ret < 0) {
> >> -            fprintf(stderr, "Hyper-V %s is not supported by kernel\n",
> >> -                    kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);
> >> +        /*
> >> +         * KVM is required to support EVMCS ver.1. as that's what 
> >> 'hv-evmcs'
> >> +         * option sets. Note: we hardcode the maximum supported eVMCS 
> >> version
> >> +         * to '1' as well so 'hv-evmcs' feature is migratable even when 
> >> (and if)
> >> +         * ver.2 is implemented. A new option (e.g. 'hv-evmcs=2') will 
> >> then have
> >> +         * to be added.
> >> +         */
> >> +        if (ret < 0 || (uint8_t)evmcs_version > 1) {

Can we start with something that won't need to be rewritten after we change the
guest evmcs version range?  e.g.:

    static bool evmcs_version_supported(uint16_t version, uint32_t supported)
    {
        uint8_t min_ver = version;
        uint8_t max_ver = version >> 8;
        uint8_t min_supported = supported;
        uint8_t max_supported = supported >> 8;
        return (min_ver >= min_supported) && (max_ver <= max_supported);
    }
    ...
    #define DEFAULT_EVMCS_VERSION ((1 << 8) | 1)
    ...
    uint16_t evmcs_version = DEFAULT_EVMCS_VERSION;
    int ret = kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0,
                              (uintptr_t)&supported_evmcs_version);
    if (ret < 0) {
        fprintf(...)
        return ret;
    }
    if (!evmcs_version_supported(evmcs_version, supported_evmcs_version)) {
        fprintf(...)
        return -ENOTSUP;
    }
    cpu->hyperv_nested[0] = evmcs_version;


> 
> > I was expecting this:
> >
> >   vcpu_evmcs_version = 1; /* hardcoded, but can become configurable later */
> >   ...
> >   kvm_vcpu_enable_cap(cs, KVM_CAP_HYPERV_ENLIGHTENED_VMCS, 0, 
> > (uintptr_t)&supported_evmcs_version);
> >   if (ret < 0 || supported_evmcs_version < vcpu_evmcs_version) {
> >     error_setg(...);
> >     return;
> >   }
> >   cpu->hyperv_nested[0] = vcpu_evmcs_version;
> >
> >
> >> +            error_report("Hyper-V %s verson 1 is not supported by kernel",
> >> +                         kvm_hyperv_properties[HYPERV_FEAT_EVMCS].desc);

s/verson/version/

> >>              return ret;

What if ret is 0?

> >>          }
> >> -
> >> -        cpu->hyperv_nested[0] = evmcs_version;
> >> +        cpu->hyperv_nested[0] = (1 << 8) | 1;
> >>      }
> >>  
> >>      return 0;
> >> -- 
> >> 2.31.1
> >> 
> 
> -- 
> Vitaly
> 

-- 
Eduardo