My unusual Linux setup of an untagged network and tagged VLANs on the same interface

December 19, 2018

One of the unusual things about my office workstation's networking is that its network interface is set up with both a regular untagged network and some VLAN-tagged networks on top. The regular untagged network is the machine's primary IP address and primary interface, on one of our main networks; the additional tagged VLAN networks are various additional internal networks here that I want my workstation to have direct access to for various reasons. I've been using this configuration for a long time now (for more than a decade), and Linux has always handled it without real problems. Basically everything just works as you'd expect; I configure the main network interface as normal, and then I add VLANs to it, originally through Fedora's network-scripts stuff and these days through systemd's networkd.

(In networkd, this means you have a .network file with a [Network] section that contains both some Address= lines for IP addresses for the interface and some VLAN= lines to define the tagged VLANs on that interface.)

This setup more or less evolved from one where I was only connected to my primary network, of course with an untagged connection. When things changed so that I could get fed tagged VLANs from an office switch, I didn't feel like redoing my machine's network configuration to be entirely tagged, so I overlaid the additional VLANs I was interested in on top of my primary network. One additional advantage of this is that in an emergency, I can connect my workstation to any untagged network port for its primary network and still keep working.

(Of course, the flipside of this is that I can't connect my workstation directly to the uplink that feeds my office switch, because that uplink carries all of our networks as tagged VLANs. And in practice my office switch's power supply failing after a power outage is the most common failure mode, so common that I have a spare stacked on top of the live switch.)

As you'd expect, all of the tagged VLANs and the main interface have the same Ethernet address. As far as I can tell this has never caused any heartburn for any of our machines, although it means that one of our firewalls sees the same MAC on more than one network.

The behavior of tcpdump in this environment is potentially a little bit inconvenient. tcpdump on the interface for a tagged VLAN works as you'd expect, but tcpdump on the main interface sees and matches all traffic, including VLAN'd traffic, and it still doesn't print the VLAN ID information by default. If I'm listening for specific traffic on my primary network, this usually doesn't matter, but if I'm listening for traffic in general (for example to spot stray broadcast traffic), I've learned to add '... not vlan' to my tcpdump expression.

(I sort of see why this is the default, but if tcpdump is going to match even VLAN'd traffic when you say, for example, 'host <IP>', I wish that by default it would print that what you were seeing is in fact a VLAN tagged packet.)

Normal Linux iptables firewall rules on the main interface don't apply to VLAN tagged traffic, as you'd expect from them being IP-level rules. I don't know if there are ways to make firewall rules that apply to VLAN tagged traffic, or just filtering rules in general.

(You can apparently do VLAN filtering on a bridge, with bridge-netfilter.)

Another drawback is that, as I've noted before, this unusual setup combined with Linux's sparse (semi-)official documentation on how the pieces of its network stack interact leaves me quite uncertain about how I might add a bridge if, for example, I wanted to use KVM based VMs that were bridged onto my machine's primary network. Fortunately VMWare's magical bridging has always just worked, both with my primary (untagged) network and with the tagged VLANs (which I have sometimes wanted to bridge VMs onto too).

PS: I call this unusual because my impression is that almost everyone either has a untagged single network or a port that's entirely tagged VLANs. I'm not even sure that all switches support this untagged plus tagged mixture on a single port, although maybe they do.


Comments on this page:

By James (trs80) at 2018-12-22 01:40:51:

It's true that some low-end switches don't support this mix of tagged and untagged, but any decent one will.

I have the same setup on my workstation, although I generally only add a VLAN interface manually on a short-term basis to configure some piece of kit.

iptables works fine with VLANs, you just have to tell it to match on the interface, eg eth0.4 for VLAN 4.

VLAN Bridges are only slightly more complicated, at least with ifupdown you just define the base bridge br0 then you can define interfaces for each tagged VLAN like br0.4. Or you can define a bridge and include tagged interfaces in it.

Written on 19 December 2018.
« My current trick for keeping reasonably ready virtual machine images
FreeBSD ZFS will be changing to be based on ZFS on Linux »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Dec 19 01:04:46 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.