WireGuard was pleasantly easy to get working behind a NAT (or several)

April 13, 2019

Normally, my home machine is directly connected to the public Internet by its DSL connection. However, every so often this DSL connection falls over, and these days my backup method of Internet connectivity is that I tether my home machine through my phone. This tethering gives me an indirect Internet connection; my desktop is on a little private network provided by my phone and then my phone NAT's my outgoing traffic. Probably my cellular provider adds another level of NAT as well, and certainly the public IP address that all of my traffic appears from can hop around between random IPs and random networks.

Most of the time this works well enough for basic web browsing and even SSH sessions, but it has two problems when I'm connecting to things at work. The first is that my public IP address can change even while I have a SSH connection present (but perhaps not active enough), which naturally breaks the SSH connection. The second is that I only have 'outside' access to our servers; I can only SSH to or otherwise access machines that are accessible from the Internet, which excludes most of the interesting and important ones.

Up until recently I've just lived with this, because the whole issue just doesn't come up often enough to get me to do anything about it. Then this morning my home DSL connection died at a fairly inopportune time, when I was scheduled to do something from home that involved both access to internal machines and things that very much shouldn't risk having my SSH sessions cut off in mid-flight (and that I couldn't feasibly do from within a screen session, because it involved multiple windows). I emailed a co-worker to have them take over, which they fortunately were able to do, and then I decided to spend a little time to see if I could get my normal WireGuard tunnel up and running over my tethered and NAT'd phone connection, instead of its usual DSL setup. If I could bring up my WireGuard tunnel, I'd have both a stable IP for SSH sessions and access to our internal systems even when I had to use my fallback Internet option.

(I won't necessarily have uninterrupted SSH sessions, because if my phone changed public IPs there will be a pause as WireGuard re-connected and so on. But at least I'll have the chance to have sessions continue afterward, instead of being intrinsically broken.)

Well, the good news is that my WireGuard setup basically just worked as-is when I brought it up behind however many layers of NAT'ing are going on. The actual WireGuard configuration needed no changes and I only had to do some minor tinkering with my setup for policy-based routing (and one of the issues was my own fault). It was sufficiently easy that now I feel a bit silly for having not tried it before now.

(Things would not have been so easy if I'd decided to restrict what IP addresses could talk to WireGuard on my work machine, as I once considered doing.)

This is of course how WireGuard is supposed to work. Provided that you can pass its UDP traffic in both ways (which fortunately seems to work through the NAT'ing involved in my case), WireGuard doesn't care where your traffic comes from if it has the right keys, and your server will automatically update its idea of what (external) IP your client has right now when it gets new traffic, which makes everything work out.

(WireGuard is actually symmetric; either end will update its idea of the other end's IP when it gets appropriate traffic. It's just that under most circumstances your server end rarely changes its outgoing IP.)

I knew that in theory all of this should work, but it's still nice to have it actually work out in practice, especially in a situation with at least one level of NAT going on. I'm actually a little bit amazed that it does work through all of the NAT magic going on, especially since WireGuard is just UDP packets flying back and forth instead of a TCP connection (which any NAT had better be able to handle).

On a side note, although I did everything by hand this morning, in theory I could automate all of this through dhclient hook scripts, which I'm already using to manage my resolv.conf (as covered in this entry). Of course this brings up a little issue, because if the WireGuard tunnel is up and working I actually want to use my regular resolv.conf instead of the one I switch to when I'm tethering (without WireGuard). Probably I'm going to defer all of this until the next time my DSL connection goes down.

Comments on this page:

I'm not 100% clear, was this your setup?

(1) Client behind NAT(s)

(2) Destination on a public IP

If so: yes, basically any VPN software will work, provided you initiate your connection from inside the NAT.

Think of it like a web browser. You can initiate a connection out of the NAT to a public IP and you now have a full two-way connection. Server streams you data, you stream it data, everyone is happy.

Sometimes firewall policies ontop of NATs get in the way, like blocking UDP or killing long-lived connections. Any good VPN software should be able to to deal with both (I know tinc does so automatically, I presume wireguard will too).

Source: having to deal with insane deep-header-inspection firewalls that drop your connection within the first four newlines or x number of bytes if it doesn't look like a permitted protocol :) Three concentric layers of TCP is bliss!

Sidenote: In a pinch: openSSH actually has layer 2 (and maybe layer 3?) VPN functionality built in. Unfortunately you need to authenticate as root on the remote end, which severely limits the situations I feel happy to use it.

By cks at 2019-04-14 19:02:36:

That was my setup; my home desktop (client) was behind my phone's NAT, while my work desktop has its usual public IP. I guess it's not too surprising that modern NATs let through bidirectional UDP traffic even when they don't recognize what it is; there's probably a lot of it from various programs that people want to run.

(Allowing outbound UDP is not a surprise; the vague surprise is allowing and NAT'ing the reverse traffic from the server to my client.)

I don't think WireGuard does anything to get around annoying UDP-blocking firewalls, at least so far. If you can't pass UDP traffic with fixed ports in both directions, I think it just fails and you have to work out some other solution.

I was going to say that the OpenSSH VPN stuff is sufficiently interesting that I should look into it but having skimmed the documentation I suspect it's not, because it looks like it'd require various pieces of setup on the server side. If I have to do server-side setup, I might as well set up a WireGuard environment instead.

(The server side isn't just permissions, it's also eg NAT'ing or otherwise handling traffic and so on.)

By Zrkr at 2019-04-15 11:05:35:

Since you didn't mention it, is there a reason why using mosh is not an option?

It survives unstable connections very well, but I suppose the fact that it requires a server-side binary to do so might be the reason why you can't use it.

By cks at 2019-04-15 11:20:28:

The smaller reason for not using mosh is that we don't have it installed on our servers at the moment. The larger reason is that mosh alone wouldn't give me access to anything that we don't expose to the Internet; it would only give me a more reliable phone-based connection to things that I can already SSH to from outside, and that's not enough in many cases.

(Pretty much the only machines we expose to the Internet for SSH are machines that general users can and are expected to log in to, which fences off things like our IMAP server, our mail server, and so on.)

(Allowing outbound UDP is not a surprise; the vague surprise is allowing and NAT'ing the reverse traffic from the server to my client.

Yes, it's done for both TCP and UDP. Even though UDP is supposed to be stateless a NAT can still surmise a 'flow' well enough to work out what to do. IIRC the flow gets deleted if no outbound UDP packets occur for more than X time, which is why many protocols send 'heartbeat' packets occasionally.

If in doubt: think of video games. Most require two-way UDP just to be able to play online. Games are often at the forefront of having to deal with networking problems, and people would complain heavily if they couldn't play their favourite FPS/RTS/MOBA/RPG/etc from behind a home or mobile NAT.

Written on 13 April 2019.
« Getting (and capturing) spam can sometimes be useful to see what's in it
Remembering that Prometheus expressions act as filters »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Apr 13 00:16:23 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.