Re: [Qemu-devel] RFC: Universal encryption on QEMU I/O channels

[Daniel P. Berrange]

On Wed, Feb 04, 2015 at 11:32:29AM +0000, Daniel P. Berrange wrote:
> In QEMU there are a number of features which involve communication with an
> external system over an I/O channel of some form. The features include
> migration, NBD, VNC and character devices. The I/O channel in question might
> might be a FIFO pipe, a PTY, a TCP socket, a UNIX domain socket, RMDA channel
> or something else, while the external system can be another QEMU, libvirt, an
> NBD server, or something else.

[snip]

> So back to QEMU/KVM, we want to have
> 
>  - Native support for encryption & authentication with migration, since
>    tunnelling via libvirt has a non-negligble performance overhead adding
>    both latency and CPU load
> 
>  - Native support for encryption & authentication with NBD server to enable
>    security of the block migration service
> 
>  - Native support for encryption & authentication with chardev TCP backends
>    to enable security of the serial port consoles.
> 
> As a starting point, the desire is to have TLS for session encryption and
> x509 certificate verification for authentication, but at a later date we'd
> also like to add the ability to use SASL for either aspect too. Thinking about
> QEMU users in general, it would probably be useful if any impl could allow for
> future enhancements such as SSH tunnelling too.
> 
> I have some development cycles available to work on addressing these gaps in
> QEMU, along with relevant experiance since I did the previous VNC server work
> for adding TLS and SASL in QEMU, along with similar in libvirt and GTK-VNC.
> 
> Having looked at the QEMU code for VNC, migration and chardevs I think the
> main stumbling block is currently that QEMU does not have any kind of standard
> approach for dealing with I/O channels internally. Migration has the QEMUFile
> abstraction, but it is currently fairly coupled to the migration code itself
> and limited in scope, because it doesn't actually deal with socket listen
> or connect tasks. That's still part of the migration code itself, QEMUFile
> only deals with I/O transfer, so it is a pretty incomplete abstraction layer.
> The chardev code has a backend abstraction, but that is really horribly
> entwined with the chardev code so not reusable in any way without a rewrite
> from scratch IMHO. The VNC server has alot of useful code for dealing with
> TLS & SASL, but it is somewhat entwined with the VNC server. The VNC server
> doesn't use any I/O abstraction just preferring raw FDs / sockets. I've not
> looked at the NBD code in detail, but I'm presuming it is using raw FDs /
> sockets.
> 
> Since this TLS/SASL code is non-trivial (and obviously security critical), I
> really don't want to end up with 4 separate places to implement it in QEMU.
> IMHO the only practical / sensible approach is to define some kind of standard
> I/O channel API internally to QEMU which migration, NBD, chardev and VNC all
> use. That gives us a single place to integrate all the security mechanisms
> we need to support.  In libvirt we did something like this a little while
> ago by defining a standard internal sockets API[1], with plugins for things
> like SASL[2], TLS[4] and SSH[5] and it has been very successful in simplifying
> the code by centralizing the hairy logic, though I wouldn't aim for exactly
> the same design if doing it again - a more layered approach like QEMU blockdev
> drivers is probably better in retrospect.
> 
> So my idea would be that we define a QEMUChannel object and set of APIs to
> standardize all interaction with sockets, pipes, RDMA, whatever $channel,
> and then convert the QEMU features I've mentioned over to use that. I think
> that would be simpler than trying to untangle QEMUFile code from migration
> and then extend its features.
> 
> A rough plan of attack would be to split the work in a number of distinct
> pieces
> 
>  - Define a basic QEMUChannel object & APIs.
> 
>  - Convert chardev backend to the QEMUChannel API
> 
>  - Convert migration to the QEMUChannel API
> 
>  - Convert NBD to the QEMUChannel API
> 
>  - Introduce support for TLS to the QEMUChannel object
> 
>  - Introduce support for SASL to the QEMUChannel object
> 
>  - Convert VNC server to the QEMUChannel API
> 
>  - Integrate TLS in migration  (eg flags turn it on/off via CLI/monitor)
> 
>  - Integrate TLS in NBD (eg flags to turn it on/off via CLI/monitor)
> 
>  - Integrate TLS in chardev backend  (eg flags to turn it on/off via 
> CLI/monitor)
> 
> Aside from the first task, I'm not sure that's the best/right order to do
> the work, it is merely the rough set of pieces I see as required. I've also
> no idea how long this would take. It is clearly a pretty large task given
> the different areas of code it touchs.

FYI, although I've been quiet on the list since this thread, I have
actually been working on this proposal. Since we ruled the use of GIO
due to the v recent min library version it would require, I looked at
the option of using the GIOChannel framework. In the end I decided
that was a dead-end since its API is rather poorly designed for
extensibility. Extending it would be more work than ignoring it,
and be less maintainable due to the need to rely on undocumented
glib hacks. So I've been moving forward with a design that is
inspired by GIOChannel, but using struct iovec to get the efficient
I/O path migration requires, and without all the charset encoding
stuff we have no need for. This has the added benefit that it lets
us fully integrate with QOM Object model and the Error object
error reporting.

So far I've

 - Defined QIOChannel base object interface
 - Implemented a QIOChannelSocket subclass for sockets
 - Implemented a QIOChannelFile subclass for pipes, devices, files, etc
 - Implemented a QIOChannelTLS subclass for the TLS protocol
 - Converted VNC to use QIOChannelSocket, QIOChannelTLS & QIOChannelWebsock
 - Converted chardev to use QIOChannelSocket & QIOChannelFile

Using these new APIs is having a dramatic positive effect on the VNC
server code, allowing all the CONFIG_* conditionals to be removed and
greatly simplifying the code.

My intention is to get the chardev backend converted and supporting TLS
encryption and websockets encapsulation before posting it for formal
review. To get there I need to

 - Finish a QIOChannelWebsock subclass for websockets protocol
 - Finish converting VNC server to new APIs
 - Define some kind of generic '-tls' command line arg for setting
   up TLS x509 credentials to be shared by -chardev & -vnc backends
 - Integrate QIOChannelTLS and QIOChannelWebsock into the chardev
   backends where appropriate (TCP chardev only most likely)
 - Finish Win32 portability of QIOChannelSocket|File subclasses

This will likely be a few weeks out from now at least. If people really
desperately want to see the work-in-progress

   https://github.com/berrange/qemu/commits/qemu-io-channel-4

Bear in mind that git repo is just a daily snapshot which crashes
in several places, doesn't yet compile on Win32, clubs baby seals,
etc, etc.

Once the first stage is posted & positively reviewed, I'll move on
to the migration client & server code and then the NBD client &
server code, converting both to use QIOChannel and thus (trivially)
gain TLS support.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|