Re: [Qemu-devel] [PATCH v2 0/6] Add netmap backend offloadings support

[Stefan Hajnoczi]

On Thu, Jan 16, 2014 at 04:00:36PM +0100, Vincenzo Maffione wrote:
> 2014/1/16 Stefan Hajnoczi <address@hidden>
> 
> > On Tue, Jan 14, 2014 at 11:59:44AM +0100, Vincenzo Maffione wrote:
> > > (3) There is actually an important problem. In the previous patch
> > version, TCP/UDP traffic was
> > >     supported between two guests attached to a VALE switch if and only
> > if both guests use (or
> > >     don't) the same offloadings: e.g. two virtio-net guests or two e1000
> > guests. This is why
> > >     in this version I put the commit which adds the support for netmap
> > as the last one and
> > >     I temporarily disable the offloading feature (e.g.
> > netmap_has_vnet_hdr() returns false).
> > >     We are working at adding proper support also in the general case in
> > which there is
> > >     a mismatch between the NIC offloadings capabilities. As soon as we
> > have proper support,
> > >     we can enable it in net/netmap.c.
> > >     What do you think about that? What's the best way to manage this
> > temporary lack of
> > >     support?
> >
> > What you described is known problem in QEMU.  You cannot unplug a
> > vnet_hdr-enabled tap netdev and plug in a non-vnet_hdr socket netdev
> > since the socket netdev and its remote side don't understand vnet_hdr.
> > This has stopped us from supporting changing NetClient peers at runtime.
> >
> > When this issue was raised we figured we'll have to add code to QEMU to
> > emulate the offload in software (i.e. TSO, checksums, etc).  But no one
> > has implemented that yet (although vmxnet3 has VMware offload software
> > emulation IIRC).
> >
> > So maybe the answer is to finally implement vnet_hdr offload emulation
> > inside QEMU?  Then netmap can negotiate with its remote side and fall
> > back to offload emulation if the remote side doesn't understand
> > vnet_hdr.
> >
> > Keep in mind that virtio-net does not allow the host to disable an
> > offload feature that was already enabled, except after the device has
> > been reset.  This precludes a simple solution where we just tell the
> > guest to stop using vnet_hdr.
> >
> 
> So what you are saying is that once you turn on an offload feature, it's
> not possible to turn it off (apart from device reset).

Yes.

> By the way, let me ask a "sidechannel" question.
> It's not clear to me how the virtio-net device decides which features to
> enable and which features not. I guess there is a negotiation between the
> guest virtio-net driver and the QEMU virtio-net frontend, and this is good.
> But if I am not mistaken there is not a negotiation between the virtio-net
> frontend and the netdev (the backend), apart from the UFO feature (I see a
> tap_has_ufo() function). This means that the virtio-frontend will enable
> the feature negotiated with the guest driver regardless of what the backend
> is able to do. This is ok as long as only tap provides those offloadings,
> but don't you see the need for a more general negotiation also between
> netdev and the frontend?e.g. extend tap_has_ufo() to all the other
> offloadings?

Here is the code:

  if (!peer_has_vnet_hdr(n)) {
      features &= ~(0x1 << VIRTIO_NET_F_CSUM);
      features &= ~(0x1 << VIRTIO_NET_F_HOST_TSO4);
      features &= ~(0x1 << VIRTIO_NET_F_HOST_TSO6);
      features &= ~(0x1 << VIRTIO_NET_F_HOST_ECN);

      features &= ~(0x1 << VIRTIO_NET_F_GUEST_CSUM);
      features &= ~(0x1 << VIRTIO_NET_F_GUEST_TSO4);
      features &= ~(0x1 << VIRTIO_NET_F_GUEST_TSO6);
      features &= ~(0x1 << VIRTIO_NET_F_GUEST_ECN);
  }

  if (!peer_has_vnet_hdr(n) || !peer_has_ufo(n)) {
      features &= ~(0x1 << VIRTIO_NET_F_GUEST_UFO);
      features &= ~(0x1 << VIRTIO_NET_F_HOST_UFO);
  }

It is assumed that most offloads are supported if vnet_hdr is available.
Only UFO is a separate feature on top of vnet_hdr.

> > I don't want to merge the tap -> net API changes and netmap offload
> > enablement until there is a solution to this.
> >
> >
> Ok, you pointed out a possible solution, but at the moment I don't think is
> easy/convenient to add negotiation support to netmap (for example, in the
> VALE switch there are more than two ports, so many possible pairs to keep
> track...).
> 
> I'm currently working on adding offloading support inside netmap (so, no
> QEMU code) and this will be enough to fix the problem for netmap, meaning
> that two arbitrary netmap clients will be able to communicate regardless of
> wether they understand or not the virtio-net-header, exactly how the tap
> (and the associated in-kernel bridge) is currently able to do. Will you
> accept the patches provided I complete the offloading support?

Yes.

> As a further step, I could try to convert the offloading code for QEMU, so
> that you can solve the socket problem you described, regardless of netmap.
> 
> Does this sound good to you?

Yes.

Stefan