My unusual Linux setup of an untagged network and tagged VLANs on the same interface
One of the unusual things about my office workstation's networking is that its network interface is set up with both a regular untagged network and some VLAN-tagged networks on top. The regular untagged network is the machine's primary IP address and primary interface, on one of our main networks; the additional tagged VLAN networks are various additional internal networks here that I want my workstation to have direct access to for various reasons. I've been using this configuration for a long time now (for more than a decade), and Linux has always handled it without real problems. Basically everything just works as you'd expect; I configure the main network interface as normal, and then I add VLANs to it, originally through Fedora's network-scripts stuff and these days through systemd's networkd.
(In networkd, this means you have a
.network file with a
section that contains both some
Address= lines for IP addresses
for the interface and some
VLAN= lines to define the tagged VLANs
on that interface.)
This setup more or less evolved from one where I was only connected to my primary network, of course with an untagged connection. When things changed so that I could get fed tagged VLANs from an office switch, I didn't feel like redoing my machine's network configuration to be entirely tagged, so I overlaid the additional VLANs I was interested in on top of my primary network. One additional advantage of this is that in an emergency, I can connect my workstation to any untagged network port for its primary network and still keep working.
(Of course, the flipside of this is that I can't connect my workstation directly to the uplink that feeds my office switch, because that uplink carries all of our networks as tagged VLANs. And in practice my office switch's power supply failing after a power outage is the most common failure mode, so common that I have a spare stacked on top of the live switch.)
As you'd expect, all of the tagged VLANs and the main interface have the same Ethernet address. As far as I can tell this has never caused any heartburn for any of our machines, although it means that one of our firewalls sees the same MAC on more than one network.
The behavior of
tcpdump in this environment is potentially a
little bit inconvenient.
tcpdump on the interface for a tagged
VLAN works as you'd expect, but
tcpdump on the main interface
sees and matches all traffic, including VLAN'd traffic, and it
still doesn't print the VLAN ID information by default. If I'm listening for specific traffic
on my primary network, this usually doesn't matter, but if I'm
listening for traffic in general (for example to spot stray broadcast
traffic), I've learned to add '
... not vlan' to my
(I sort of see why this is the default, but if
tcpdump is going
to match even VLAN'd traffic when you say, for example, '
<IP>', I wish that by default it would print that what you were
seeing is in fact a VLAN tagged packet.)
Normal Linux iptables firewall rules on the main interface don't apply to VLAN tagged traffic, as you'd expect from them being IP-level rules. I don't know if there are ways to make firewall rules that apply to VLAN tagged traffic, or just filtering rules in general.
(You can apparently do VLAN filtering on a bridge, with bridge-netfilter.)
Another drawback is that, as I've noted before, this unusual setup combined with Linux's sparse (semi-)official documentation on how the pieces of its network stack interact leaves me quite uncertain about how I might add a bridge if, for example, I wanted to use KVM based VMs that were bridged onto my machine's primary network. Fortunately VMWare's magical bridging has always just worked, both with my primary (untagged) network and with the tagged VLANs (which I have sometimes wanted to bridge VMs onto too).
PS: I call this unusual because my impression is that almost everyone either has a untagged single network or a port that's entirely tagged VLANs. I'm not even sure that all switches support this untagged plus tagged mixture on a single port, although maybe they do.