The easy way to wind up with multiple subnets on a single (V)LAN segment

March 10, 2013

In theory, a nice proper network is supposed to have only a single IP(v4) subnet running over any given network segment. But this is not actually fully required; if you're sufficiently perverse you can run multiple subnets over the same physical network. Some people are now asking how on earth you would ever get into such a crazy situation. Well, sit back, I have a story for you.

Suppose that you use private subnets and in particular you put each group in your organization in its own subnet (around here, a group is a research group). Often when you start doing this, you will give a group a /24 (because that's the simple approach). But the thing about groups is that sometimes they, well, grow. Some energetic professor will get a grant for some equipment here and another one will get a grant for a cluster there and before you know it, a /24 just isn't enough space.

In a perfectly ideal world you would have allocated all those initial /24s such that they could grow substantially, at least into /22s. In a less than ideal world you just allocated the /24s sequentially and there is no room to expand them. No problem; private IP address space is capacious, so you can just start over by giving a group, say, a /16. But this new /16 is not contiguous with the group's old /24; you can't just expand the segment's subnet mask and be done.

In an ideal world you could arrange with the group to have a flag day for the changeover between their old subnet and their new subnet; on that day, all machines would go down, all IP addresses would be changed, and the group would completely migrate from their /24 to their /16. In the real world the group is going to laugh politely at you when you propose this, because they (rightfully) have much more important things to do with their time than go through a large disruption just for your convenience. Instead you're forced into the obvious step: you just add the new /16 to their existing network segment alongside the old /24. The group will put new machines in the /16 and migrate old machines from the old /24 to the new /16 at their convenience (generally very slowly).

This does have some small drawbacks. The largest one is that all of the group's traffic between their two subnets is making an unnecessary round trip through your router (and if your router is a firewall, through your firewall rules; you'll wind up making a special exemption for their internal traffic). Hopefully there won't be much of it; if there is, you can sometimes use it to motivate the group into moving some machines.

(The obvious workaround for heavy-traffic machines is to give them a second IP address alias on the other subnet, so they know that they can reach it directly.)

Sidebar: easy IP address assignment in the new /16

We generally suggest to groups that the low /24 of the new /16 be reserved for a one to one mapping of machines in the old /24. Groups don't necessarily have to convert (or dual-home) machines this way, often they renumber them during a move in order to (re)organize their network, but it makes it easy to do a quick conversion or a dual-homing; just change the network without changing the host IP.

Comments on this page:

From at 2013-03-11 11:18:00:

"drawbacks... unnecessary round trip through your router."

I believe routers react to this by sending an ICMP redirect, which will cut out the unnecessary round-trip.

I think I've seen it on a Linux router (and working with Linux hosts). Googling ICMP redirect also returned documentation that Cisco routers do it.

By cks at 2013-03-11 12:36:48:

Although I may be wrong on this, I don't think that the router can usefully send an ICMP redirect. While the two subnets are on the same physical network segment they are still separate subnets; without special hacks to get machines to believe that a detached subnet is still directly reachable, I think IP stacks are reasonably going to insist on there being some router in the process.

(This does make me wonder if you can persuade a machine that a subnet is directly reachable on a directly attached network segment without the machine having an IP address on the subnet. Sadly I lack a suitable test environment right now, but it looks like Linux's ip route command will at least accept it. On the other hand I have no idea what the receiving machines will make of the resulting ARP requests, since they have an off-network source IP from their perspective.)

From at 2013-03-11 13:18:43:

Good point.

I picked up this idea while investigating "wireless isolation" on my home AP. I'm pretty sure I provoked the router to send an ICMP redirect to an on-link address.

More Google says ICMP redirects used to be (ab)used to achieve this, but that the wording of an RFC ruled it out 15 years ago.

Written on 10 March 2013.
« The systemd dependency problem
In universities, computers are not an essential service »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Mar 10 23:05:42 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.