The problem of network tunnels and (asymmetric) routing

September 16, 2022

Let's suppose that you have an inside network I on which you have a bunch of things used in your environment, like central syslog servers, your client mail gateway machine, and so on, and you also have an external machine E, that you would like to be able to use those services. One obvious seeming way that you could do this is by setting up some form of network tunnel between E and a touchdown machine T that has access to your inside network I (these days you might use WireGuard, for example). Your exterior machine E sets a network route to I that goes through the tunnel, the traffic pops out from T (behind your perimeter firewall), and everything is happy, right?

Unfortunately, often not so much these days, because of stateful firewalls. The problem is that the nice simple model I've described here has asymmetric routing. E's packets to the inside network I goes via the tunnel and T, but under normal circumstances the return traffic from machines on I will go out their regular network gateway, not back via T. If there is a stateful firewall between your I machines and E, it may well become unhappy and start blocking this traffic, since it looks like a half-open connection (since it's only seeing half of the traffic).

One answer to this is to have the tunnel endpoint T be a NAT gateway, not just a simple tunnel endpoint that passes traffic in the clear; then E's traffic to machines on I emerges with T's IP address on it, so return packets go back through T. However this leaves you with a different asymmetric routing problem if machines on I ever reach out to E on their own (for example to SSH in to it or to collect metrics from it). Their packets will flow out normally, un-NAT'd, but E will try to send packets for them back through the tunnel and T's NAT'ing. You can solve this with simple "policy based" routing on E, so that reply packets go out the interface they came in on.

(You can also solve this by having E only route through the tunnel for machines on I that never reach out to it, setting host routes instead of network routes, although this is potentially fragile.)

Another solution is to teach all machines on your internal network I that the external machine E is actually reached through a special route to the tunnel gateway T. If T is not on I itself and is reached through the same router as the default route, you might be able to do this by a change to your gateway router alone. The obvious drawback to this is that now the tunnel gateway T becomes an additional point of failure for reaching E (well, for machines on the internal network I).

(In a sufficiently complex environment, this can be automated through routing announcements; T and your collection of other tunnel gateways all announce what IP addresses are reachable through them, and various things listen and respond appropriately. When a tunnel is down, the routes are withdrawn and things go back to the defaults.)

Your life is generally easier if the external machine E is not directly reachable from the Internet and instead you have to go through a different IP address to reach it (such as a gateway); often this means you have no conflicting routes, since you can't reach E's (private) IP address except through the tunnel. Of course you may have a similar problem if you need to manage the gateway machine itself, since that machine definitely has a public IP address.

Comments on this page:

From at 2022-09-17 06:37:29:

Those solutions are not mutually exclusive; even if E has a public IP address, the tunnel interface typically has its own private IP prefix assigned (which I's gateways know about), giving you both options at once – E will use its private wg0 address as source for all traffic that it sends through the tunnel (e.g. to the syslog server), so it's automatically the case that the reply traffic follows I's internal routing and eventually makes its way back to the tunnel, while connections from I to E (or from the outside world to E) can still use the latter's public address without relying on the tunnel.

But besides that, routing announcements don't need a minimum level of complexity (i.e. you don't have to deal with BGP); e.g. Bird2 for OSPF is fairly low-effort and can be run between E and T1/T2 over tunnels, and/or between the core gateway and T as needed. (In my case, the main gateway for I is T, so it just automatically knows the routes.)

(Another option for redundant T1/T2 is having the core gateway have a static route via T's VRRP/CARP address; I've never gotten around to trying keepalived on Linux but I believe OpenBSD has CARP built in.)

Written on 16 September 2022.
« The C free() API gives libraries and functions useful freedom
Authenticated SMTP and IMAP authentication attacks and attempts we see here »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Sep 16 22:45:27 2022
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.