Re: [Qemu-devel] libvirt/QEMU/SEV interaction

[Relph, Richard]

A few answers in line…

On 9/8/17, 8:16 AM, "Daniel P. Berrange" <address@hidden> wrote:

    On Fri, Sep 08, 2017 at 06:57:30AM -0500, Brijesh Singh wrote:
    > Hi All,
    > 
    > (sorry for the long message)
    > 
    > CPUs from AMD EPYC family supports Secure Encrypted Virtualization (SEV)
    > feature - the feature allows running encrypted VMs. To enable the feature,
    > I have been submitting patches to Linux kernel [1], Qemu [2] and OVMF [3].
    > We have been making some good progress in getting patches accepted 
upstream
    > in Linux and OVMF trees. SEV builds upon SME (Secure Memory Encryption)
    > feature -- SME support just got pulled into 4.14 merge window. The base
    > SEV patches are accepted in OVMF tree -- now we have SEV aware guest BIOS.
    > I am getting ready to take off "RFC" tag from remaining patches to get 
them
    > reviewed and accepted.
    > 
    > The boot flow for launching an SEV guest is a bit different from a typical
    > guest launch. In order to launch SEV guest from virt-manager or other
    > high-level VM management tools, we need to design and implement new
    > interface between libvirt and qemu, and probably add new APIs in libvirt
    > to be used by VM management tools. I am new to the libvirt and need some
    > expert advice while designing this interface. A pictorial representation
    > for a SEV guest launch flow is available in SEV Spec Appendix A [4].
    > 
    > A typical flow looks like this:
    > 
    > 1. Guest owner (GO) asks the cloud provider to launch SEV guest.
    > 2. VM tool asks libvirt to provide its Platform Diffie-Hellman (PDH) key.
    > 3. libvirt opens /dev/sev device to get its PDH and return the blob to the
    >    caller.
    
    What sort of size are we talking about for the PDH ?

The PDH blob is described in reference 4. It’s 0x824 bytes long… a bit over 2K 
bytes.
PDH is “Platform Diffie-Hellman” key, public portion.
    
    There's a few ways libvirt could report it
    
     1. As an XML element in the host capabilities XML
     2. As an XML element in the emulator capabilities XML
     3. Via a newly added host API
    
    > 4. VM tool gives its PDH to GO.
    > 5. GO provides its DH key, session-info and guest policy.
    
    Are steps 4 & 5 strictly required to be in this order, or is it
    possible

Steps 4 and 5 must occur in that order. The data sent by the GO in the “session 
info” is encrypted and integrity protected with keys that the GO derives from 
the GO private Diffie-Hellman key and the PDH public Diffie-Hellman key.
    
    What are the security requirements around the DH key, session info
    and guest policy ? Are any of them sensitive data which needs to be
    kept private from untrustworthy users. Also are all three of these
    items different for every guest launched, or are some of them
    likely to be the same for every guest ?

The PDH is not sensitive. It is relatively static for the platform. (It only 
changes when the platform owner chooses to change the apparent identity of the 
platform that will actually run the virtual machine.)
The 128-byte session info data is encrypted and integrity protected by the GO. 
It need not be additionally encrypted or integrity protected. It should vary 
for EVERY guest launch.
The 4-byte guest policy must be sent in the clear as some components may want 
to observe the policy bits. It may change from guest to guest, but there will 
likely only be a few common values.
    
    eg, would the same guest policy blob be used for every guest, with
    only session info changing ?

Also what sort of size are we talking about for each of these
    data items, KBs, 10's of KB, 100's of KBs or larger ?
    
    The security and data size can influence our design approach from
    the libvirt POV.
    
    > 6. VM tool somehow communicates the GO provided information to libvirt.
    
    Essentially we have two choices
    
     1. inline in the guest XML config description passed to libvirt
        when defining the guest XML
    
     2. out of band, ahead of time, via some other API prior to defining
        the guest XML, which is then referenced in guest XML. THis could
        be done using the virSecret APIs, where we create 3 secrets, one
        each for DH key, session-info and guets policy. The UUID of the
        secret could be specified in the guest XML. This has flexibility
        of allowing the same secrets ot be used for many guests (if this
        is valid for SEV)
    
    
    > 7. libvirt adds "sev-guest" object in its xml file with all the 
information
    >    obtained from #5
    > 
    >    (currently my xml file looks like this)
    > 
    >    <qemu:arg value='-object'>
    >    <qemu:arg
    > 
value='sev-guest,id=sev0,policy=<GO_policy>,dh-key-file=<filename>,session-file=<filename>/>
    >    <qemu:arg value='-machine'/>
    >    <qemu:arg value='memory-encryption=sev0'/>
    > 
    > 8. libvirt launches the guest with "-S"
    
    All libvirt guests get launched with -S, to give libvirt chance to do some
    setup before starting vCPUs. Normally vCPUs are started by default, but
    the VIR_DOMAIN_START_PAUSED flag allows the mgmt app to tell libvirt to
    leave vCPUS paused.
    
    Alternatively, if libvirt sees presencese of 'sev' config for the guest,
    it could automatically leave it in PAUSED state regardless of the
    VIR_DOMAIN_START_PAUSED flag.

While using the LAUNCH_MEASURE and LAUNCH_SECRET operations is expected to be 
the common case, they are optional. I would recommend requiring the upstream 
software to explicitly set the VIR_DOMAIN_START_PAUSED flag, if only to 
minimize the dependencies…
    
    > 9. While creating the SEV guest qemu does the following
    >  i) create encryption context using GO's DH, session-info and guest policy
    >     (LAUNCH_START)
    >  ii) encrypts the guest bios (LAUNCH_UPDATE_DATA)
    >  iii) calls LAUNCH_MEASUREMENT to get the encrypted bios measurement
    > 10. By some interface we must propagate the measurement all the way to GO
    >   before libvirt starts the guest.
    
    Again, what kind of size data are we talking about for athe "measurement"
    blob ? a KB, 10's of KB, or more ?

The measurement is 48 bytes…
    
    My first gut instinct would be for QEMU to emit a QMP event when it has
    the measurement available. The event could include the actual data blob,
    or we can could add an explicit QMP command to fetch the data blob.
    
    Libvirt could listen for this QEMU event, and in turn emit an event from
    libvirt with the same data, which the mgmt tool can finally give to the
    GO.
    
    > 11. GO verifies the measurement and if measurement matches then it may
    >  give a secret blob -- which must be injected into the guest before
    >  libvirt starts the VM. If verification failed, GO will request cloud
    >  provider to destroy the VM.
    
    So we need some mechanism to provide the secret blob.  This could be
    done via a new libvirt API.  Alternatively, if we're using virSecret
    for the other stuff, the guest config XML could include the UUID of
    a 4th  secret. Libvirt would then watch to see when that secret has
    a value set, and pass that onto QEMU.

The secret is optional… it is up to 16KB, already encrypted, and integrity 
protected with an IV and MAC value passed with the secret. The guest address 
the secret should be deposited at is 
Technically, the SEV FW API allows there to be more than one secret… I don’t 
see a strong reason to require libvirt to support more than one, though.
    
    > 12. After secret blob is injected into guest, we call LAUNCH_FINISH
    >   to destory the encryption context.
    > 13. libvirt issues "continue" command to resume the guest boot.
    
    This is as simple as the mgmt tool calling virDomainResume to unpause
    CPUs. We could have it such that virDomainResume checks whether the
    virSecret has been populated secret blob by GO, and pass it onto
    QEMU at this time.
    
    > Please note that the measurement value is protected with transport
    > encryption key (TIK) and it changes on each run. Similarly the secret blob
    > provided by GO does not need to be protected using libvirt/qemu APIs. The
    > secret is protected by TIK. From qemu and libvirt point of view these are
    > blobs and must be passed as-is to the SEV FW.
    > 
    > Questions:
    > a) Do we need to add a new set of APIs in libvirt to return the PDH from
    > libvirt and VM tool ? Or can we use some pre-existing APIs to pass the
    > opaque blobs ? (this is mainly for step 3 and 6)
    > b) do we need to define a new xml tag to for memory-encryption ? or just
    > use the qemu:args tag ? (step 6)
    
    <qemu:args> is explicitly only ever for ad-hoc testing.
    
    For anything that is to be used in production deployment we must
    explicitly model it in the XML. So we definitely need new XML
    defined.
    
    > c) what existing communicate interface can be used between libvirt and 
qemu
    > to get the measurement ? can we add a new qemu monitor command
    > 'get_sev_measurement' to get the measurement ? (step 10)
    
    Yes, QMP commands seeem most likely.
    
    > d) how to pass the secret blob from libvirt to qemu ? should we consider
    > adding a new object (sev-guest-secret) -- libvirt can add the object 
through
    > qemu monitor.
    
    Yeah, that looks like a viable option too.
    
    > 
    > 
    > [1] https://marc.info/?l=kvm&m=150092661105069&w=2
    > [2] https://marc.info/?l=qemu-devel&m=148901186615642&w=2
    > [3] https://lists.01.org/pipermail/edk2-devel/2017-July/012220.html
    > [4] http://support.amd.com/TechDocs/55766_SEV-KM%20API_Specification.pdf
    
    Regards,
    Daniel
    -- 
    |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange 
:|
    |: https://libvirt.org         -o-            https://fstop138.berrange.com 
:|
    |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange 
:|