Re: [Qemu-discuss] Using multiple network interfaces on a guest

[Pavel Troller]

> On 06/17/2014 01:44 AM, Pavel Troller wrote:
>> Hi!
>>    I have a strange problem. I need to run a guest with two separate network
>> cards. On the host, their tap ends are connected to two separate bridges.
>> It's on qemu-2.0.0. Related part of the command line looks like this:
>> qemu-system-x86_64...
>> ...-net nic,model=virtio,macaddr=xx-xx-xx-xx-xx-xx -net 
>> tap,ifname=alpha-tap..
>> ...-net nic,model=virtio,macaddr=yy-yy-yy-yy-yy-yy -net 
>> tap,ifname=bravo-tap..
>> They are incorporated to the bridges on the host:
>> address@hidden:# brctl show
>> bridge name     bridge id               STP enabled     interfaces
>> br0             8000.300bff486546       no              eth1
>>                                                          delta-tap
>>                                                          bravo-tap
>>                                                          echo-tap
>> br1             8000.bc03d650794a       no              eth0
>>                                                          alpha-tap
>>
>> The br1 bridge is anonymous one, it doesn't have an IP address and serves 
>> just
>> to bridge the traffic from the guest to a dedicated physical card (eth0).
>> delta-tap and echo-tap come from another guests. The br1 bridge never 
>> contains
>> more interfaces.
>>
>> On the guest, there is no bridging between its virtual nics. Every one has 
>> its
>> own IP address and some set of routes, there is just one default route,
>> leading to the dedicated network card on the host. But NO BRIDGING. There is
>> no bridge interface activated at all.
>>
>> And now, to the problem: Immediately after bringing the guest up and adding
>> its interfaces to the appropriate bridges, it starts to bridge them together!
>> All the traffic (including arp queries, DHCP transactions etc.) visible on
>> br0 starts to be visible on br1 and vice versa. It makes a lot of serious
>> network problems - confused ARP tables, unstable routing, even forwarding
>> the local DHCP server (coming from delta-tap on br0) to the dedicated card
>> (eth0) on br1, etc. Bringing either alpha-tap or bravo-tap down stops the
>> bridging, bringing them up starts it again. Bringing down the virtual nics
>> on the guest DOESN'T stop the bridging at all.
>>
>>    I can't understand, why this happens. How to prevent the bridging ? Is it
>> a feature of tap interfaces, that all ones are automatically bridged 
>> together?
>> Or is it a qemu bug ? I'm really confused and a lot of hours spent by
>> searching the problem on the net didn't bring anything useful, it seems that
>> nobody ever tried more virtual nics on one guest.
>>    Sorry for such a long post. Any useful reply will be highly appreciated.
>>
>> With regards,
>>    Pavel
>
> tl;dr version. add 'vlan=1' to the tap and nic that go together and 
> 'vlan=2' to the other tap and nic pair. or use '-netdev' and '-device' 
> which are the more modern way of defining networking.
>
> this is expected behavior and it is qemu itself that is bridging the 
> traffic. you need to add vlan= options to each of your '-net nic' and '-net 
> tap' configuration options to separate the traffic. for example,
>
> '... -net nic,model=virtio,macaddr=xx-xx-xx-xx-xx-xx,vlan=1 -net 
> tap,ifname=alpha-tap,vlan=1,...'
> '...-net nic,model=virtio,macaddr=yy-yy-yy-yy-yy-yy,vlan=2 -net 
> tap,ifname=bravo-tap,vlan=2,...'
>
> the reasoning behind this is how qemu connects the guest network device 
> (the -net nic) with something in the host to act on the packets coming from 
> the guest (the -net tap). when the -net options are used, qemu will 
> internally create a software ethernet hub. traffic that comes in from one 
> logical port on this hub gets sprayed out to all other logical ports. the 
> vlan option can be used to specify multiple of these internal hubs and what 
> should be connected to each. by default, the vlan option is 0. this results 
> in both tap interfaces and both guest network interfaces being connected to 
> the same hub in the configuration you listed.
>
> the vlan option does not refer to 802.1q vlans in the ethernet switching 
> world but only to an internal construct in qemu. this unfortunately leads 
> to much confusion. qemu has no knowledge of 802.1q vlan tags and wont be 
> changing packets to contain vlan tags.
>
> there is a second way to solve this is by using the -device and -netdev 
> options. this method does not create the software hub by default and 
> directly connects a guest device with a host device. something like the 
> following should do it.
>
> '... -netdev tap,ifname=alpha-tap,id=network0,<any other tap options> 
> -device virtio-net-pci,netdev=network0,macaddr=xx-xx-xx-xx-xx-xx'
>
> this explicitly connects a host device, or netdev, with a guest device by 
> using a specific identifier. 'network0' in this case but the name doesn't 
> matter as long as it doesn't conflict with other names. the '-net' options 
> are just legacy front ends to this functionality internally to qemu. using 
> '-net' is actually considered deprecated in favor of the '-netdev' and 
> '-device' options.
>
> anyways, that was probably more information than you needed to accomplish 
> the task you need but hopefully gives you a good understanding of what is 
> happening and how to go about fixing it.
>
> mike

Hello Mike,
  many thanks for very detailed and comprehensive explanation! Now
I understand what's happening. I'll stick to the new -netdev syntax, it seems
to be better in other aspects as well.
  WIth regards,
    Pavel