A wish for KVM virtualization: simple bridged networking

December 11, 2009

I have a sad confession: despite running Fedora 11 on a machine that's fully capable of hardware virtualization, I am still using VMWare (and living with various bits of pain). While there are side reasons, the major one is that as far as I can see, KVM doesn't have simple bridged networking that needs no host-side changes, and VMWare does.

(Now, I freely admit that I may well be missing something in KVM and its setup. I certainly hope so; I would like to stop routinely using VMWare.)

My most common use for virtualization is to bring up test servers that are part of our overall environment. As you might expect, our environment does not expect servers to be living behind NAT gateways or the like; it expects them to live on distinct and fully reachable IP addresses. In short, they need to be bridged onto the same network that my host machine is on.

In VMWare this is reasonably simple in the GUI but apparently requires ugly kernel hacks behind the scene. It does not require any changes to my host's own networking; VMWare's ugly kernel hacks make it magically work.

In KVM, it appears that you must create a bridge and then change your host's networking to use the bridge instead of the normal network interface. This is terribly invasive, because my host networking is very complex; I have VLANs, firewall rules, and complex policy based routing all attached to my host network. It's not even clear exactly what has to move; for example, do the VLANs stay attached to the real network interface or move to the bridge?

(If you create the bridge and just attach it to your normal network interface without changing your host's networking, your host networking stops working. Or at least this is what happened when I tried it.)

Now, I can see why the kernel people consider this the technically correct solution. But, much as with Xen, I'm not willing to make that big a commitment to KVM when VMware can be used without it, especially when I'm not certain that I'll like KVM in the first place. So I really wish that KVM did have simple bridged networking, because then I could actually try it out.

(VirtualBox does seem to have simple bridged networking, but then I'm not sure that it's any friendlier to the kernel than VMWare is and I know that VMWare has a better interface.)

Sidebar: why a virtual machine network is not an option

In theory, the correct way to deal with this issue is to create a new public but virtual network for my virtual machines to sit on that my host machine 'routes' to. This keeps potentially troublesome virtual machines off our physical network while still giving them reachable IP addresses.

As theoretically elegant as this solution is, it's unworkable in practice, especially when more than one sysadmin wants to use virtualization (we use our workstations as host machines). First, adding a new reachable network is not a trivial operation in our environment, even assuming that I wanted my host machine to be configured as a (full) router. Second, this essentially precludes using public Internet IP addresses, and some of the testing that we do needs machines with such IP addresses.


Comments on this page:

From 114.134.11.1 at 2009-12-11 07:12:22:

ProxyArp?

~~~ PerryLorier
From 193.130.233.2 at 2009-12-11 07:35:28:

xen?!?

From 75.236.117.188 at 2009-12-11 18:40:11:

Everything that was on eth0 moves to br0. If you have subinterfaces on eth0:x, those go to br0:x. Other interfaces remain the same. Basically, br0 incorporates eth0. If you have a pare nic, doing the bridging there can be less hassle.

-dsr-

From 99.55.227.127 at 2009-12-11 19:55:17:

I've had this same wish. I haven't tested it, but it sounds like they've added this feature in Fedora 12. Note 5.3.7 of http://docs.fedoraproject.org/release-notes/f12/en-US/html/sect-Release_Notes-Virtualization.html says that "Commonly used host network configurations, like bridges, bonds, VLAN's and sensible combinations thereof may now be created using the general-purpose network configuration library, netcf." It sounds like you can't manage bridges via virt-manager, yet, though.

From 74.94.68.194 at 2009-12-12 21:49:21:

I personally don't use the networking built into virsh, I opt for just straight bridging.

ala

   <interface type='bridge'>
     <mac address='52:54:32:34:4b:7e'/>
     <source bridge='br0'/>
     <target dev='vnet0'/>
     <model type='virtio'/>
   </interface>

Obviously you can change 'br0' to something else, so if you want vlan123, and you had br0.123 configured... haha!

By Alex Xu (Hello71) at 2022-04-16 11:40:30:

This was actually fixed the next year with the addition of macvtap. macvtap interfaces are attached directly to ethernet interfaces; the main purpose of this is improved performance, but it has the added benefit of not requiring sed -i -e s/eth0/br0/ /etc/**. The downside is that if you want to access it via the host, you need both a macvlan and a macvtap for some reason.

Written on 11 December 2009.
« How not to copy a file to standard output in Python
You should always be able to get a program's version and basic usage »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Dec 11 02:18:01 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.