[Qemu-devel] Re: libvirt vs. in-qemu management

[Daniel P. Berrange]

On Tue, Apr 06, 2010 at 02:43:47PM +0200, Alexander Graf wrote:
> Daniel P. Berrange wrote:
> > With appliances there are two core aspects
> >
> >  1. The description of VM hardware requirements
> >  2. The disk format
> >
> > Traditionally VMware appliances have shipped a VMX file for 1. and
> > a VMDK file for 2. 
> >
> > Shipping the native config file format with an appliance though is
> > the wrong thing to be doing. The native config format describes the
> > configuration for a VM for a specific deployment. This is not the
> > same as describing the hardware requirements of an appliance. As
> > the most simple example, a native config would have hardcoded disk
> > paths, or a specific choice of host network connectivity. Neither
> > of these things have any business being in the appliance config.
> >
> > For this reason, there are now specific appliance formats. Libvirt
> > has long had its own appliance format (virt-image) which is separate
> > from the main XML format, so it avoids hardcoding deployment specific
> > options. There is also the vendor neutral OVF format which is widely 
> > supported by many mgmt tools. 
> >
> > If people want to ship QEMU appliances I don't think libvirt is 
> > causing any problems here. Simply ship a OVF description + either
> > raw or qcow2 disk image. Any app, libvirt, or not could work with
> > that.
> >   
> 
> Does VMware Player support OVF?
> Does VMware Workstation support OVF?
> Does VMware Server support OVF?

I've no idea if they're added support to those. There's no technical
reason why not, but being closed source software they may have 
artificially restricted functionality to force you to get VCenter.

> Do older VMware ESX versions support OVF?

I don't know off hand what version it was introduced in

> Does it make sense to build an OVF with a Xen PV image?

Yes, in fact it is beneficial. If you shipped a PV ops enabled
appliance image with a Xen config file, the distributor would
have to ship several configs one for PV mode, one for HVM, since
it hardcodes the type of guest & disk drivers. If you ship an 
OVF file, then the tool deploying the appliance can decide whether
to deploy it in PV or HVM mode. This same appliance can then even
work for KVM too.

> We need to deliver vendor specific configs anyways. Of course we could
> ship a VMware type, a Xen type and an OVF type. But that would certainly
> not help KVM's awareness because it's hidden underneath the OVF type.

The whole point of the OVF format/project is that it is not explicitly
targetted at one vendor's technology. This isn't the right place to be
raising awareness of KVM.

> 
> >   
> >> 3) Configuration conversion
> >>
> >> Party due to qemu not having a configuration format, partly due to 
> >> libvirt's ambivalent approach, there is always conversion in 
> >> configuration formats involved. I think this is the main reason for
> >> the feature lag. If there wasn't a conversion step, there wouldn't 
> >> be lag. You could just hand edit the config file and be good.
> >>     
> >
> > [snip]
> >
> >   
> >> Point 3 is the really tough one. It's the very basis of libvirt. And 
> >> it's plain wrong IMHO. I hate XML. I hate duplicated efforts. The 
> >> current conversion involves both. Every option added to qemu needs to 
> >> be added to libvirt. In XML. Bleks.
> >>     
> >
> > In the previous thread on this topic, I've already stated that we're
> > interested in providing a means to pass QEMU config options from
> > libvirt prior to their full modelling in the XML, to reduce, or completely 
> > eliminate any time-lag in using new features.
> >   
> 
> That would cover new features and would be really good to have
> nevertheless. It still doesn't cover the difference in configuration for
> native tags. Imagine you'd want to enable cache=none. I'd know how to do
> it in qemu, but I'd be lost in the libvirt XML. If I'd be a person
> knowledgeable in libvirt, I'd know my way around the XML tags but
> wouldn't know what they'd mean in plain qemu syntax. So I couldn't tell
> people willing to help me what's going wrong even if I wanted to.

Going from the XML to QEMU config and vica-verca is not rocket science.
You can trivially see the QEMU config generated for any libvirt VM in
the logs. There are also commands for doing the conversion in both
directions, though I admit the QEMU -> XML conversion is not as complete
as the XML -> QEMU conversion.

> If instead there was a common machine description file that everyone
> knows, there'd be a single point of knowledge. A RHEL-V admin could work
> on plain qemu. A qemu developer would feel right at home with virt-manager.

This isn't solving the problem. If you see a problem in the QEMU config
uses by a high level tool like RHEV/oVirt, you still aren't going to 
know what the config change you need to make in those apps. They are
never going to work with the QEMU config as their master data format.
It is just something they generate on the fly at runtime, from their
SQL databases, because they want to model concepts at a high level.
A VM as represented in RHEV/oVirt does not have a single QEMU or libvirt
config file description - the low level config can potentially vary each
time the guest is started on a host(s).

> > Even if an app was using QEMU directly, you can't presume that the app 
> > will use QEMU's config file as its native format. Many apps will store
> > configs in their own custom format (or in a database) and simply generate
> > the QEMU config data on the fly when starting a VM. In the same way libvirt
> > will  generate QEMU config data on the fly when starting a VM. Having many
> > config formats & conversion / generation of the fly is a fact of life no 
> > matter what mgmt system you use.
> >   
> 
> I don't see why we shouldn't try to change that. Why not generate a
> common machine description file in qemu for all qemu VMs? Think of word
> documents. Everyone knows how to read and write .doc files. Why
> shouldn't VM description files be the same? It's really the best case
> for the user if there's a single type of configuration.

The raw QEMU config for a disk device is specified in terms of the
file path for the storage.  A management app using QEMU / libvirt is
not going to store its config for the guest in this way. They will
have some model of storage and an association between a storage volume
and a virtual machine. The actual file path for this may is only relevant
at the time the VM is actually started & may be different on every host
the VM is run on. eg if you've associated a VM with a LUN based, it may
be /dev/sda when run on host A and /dev/sdz on host B. The mgmt app is
going to use a mapping based on the  WWID, not paths. 

> 
> > The key point is that it needs to be really easy to get at the generated
> > QEMU config data to see what is actually being run. libvirt will always
> > save the exact QEMU config data it generates into a log file for this
> > exact purpose (/var/log/libvirt/qemu/$VMNAME.log).  The complexity here
> > is that you can't directly run this if TAP devices are in use for the
> > networking since it'll expect TAP FDs to be passed down. Although you 
> > could allow QEMU to open the TAP devices, this is not good  for security 
> > separation of QEMU from the host OS, so I'm not sure it can be easily
> > avoided. 
> >   
> 
> Sounds like that particular feature needs to go into the qemu stack
> then, so it can be properly described in the config file.
> 
> > One attempt to make life easier, was that we added a libvirt command 
> > 'virsh domxml-to-native' command which given a libvirt XML config file
> > would return a set of QEMU command line args which *can* be used to
> > start the VM directly. This is basically identical to the normal QEMU
> > args libvirt generates but relies on the ifup script to create TAP
> > networking instead.
> >   
> 
> That helps for XML -> qemu. It doesn't help when you're trying to go the
> other way around. You'd still have to learn a different syntax. People
> are reluctant to learn new syntaxes.

In 99% of the cases you only need to know one syntax - the syntax that is
appropriate for the app you're interacting with. You only need to know
multiple syntaxes if you're trying to play at all levels of the stack
which is not something we should be expecting in the common case.


Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|