Reaching past our firewalls with WireGuard (some thoughts)

September 27, 2022

Our network design carries with it the implicit assumption that all of our machines (and all machines run by other people) are within our network perimeter, so we can safely expose dangerous services to them, like an unauthenticated SMTP 'smarthost' and a central syslog server. In the beginning this was completely true, but over time we've acquired a few machines that are outside our network perimeter (and if there is cloud in our future, we'll get more). We'd rather like for these external machines to have some access to our standard services; at a minimum we want them to be able to email us when they have issues liked failed disks, and it would be nice to be able to collect logs and so on. Our current solution to this is to poke holes through our firewall, but recently I've been tempted by the idea of using WireGuard as a more secure approach.

The appeal of WireGuard for this is that it's a lightweight service that requires little configuration or operation, and is now supported across all of our Ubuntu fleet. This creates two obvious options, depending on how much work we want to do on these external machines. The first option is to run WireGuard in a non-routed, "point to point" mode on each of the internal machines that have services we want to provide access to. The internal machine would expose its service(s) on an private WireGuard network as well as its normal IP address (in many cases this requires no service changes), and external machines would reach the service by talking to the internal machine's private WireGuard IP address. The one drawback to this is that it requires configuring each external machine to use the appropriate magic WireGuard IPs for these services, instead of the hostnames we normally use.

The other approach would be to create a NAT'ing WireGuard gateway machine and then configure external machines to route traffic to specific machines that run relevant services (our mail smarthost, our central syslog server, etc) through their WireGuard tunnel to the gateway. Because of the problem of network tunnels and asymmetric routing this mostly requires these internal machines to never want to initiate connections to the external ones (which is true for services today), and it makes the WireGuard NAT gateway machine a central point of failure for all external machines talking to all internal services. But it avoids configuring WireGuard on a bunch of internal machines and changing the service configuration on the external machines (since they can keep on using the regular hostnames for things like our SMTP smarthost).

(Not having to change the configuration of things on external hosts makes life much easier, because we don't have to keep two copies of configurations.)

You can do this (in either approach) with any VPN or secure IP tunneling system. The advantage of WireGuard is that it's very easy and lightweight, so it's simple to configure on the external machines and viable to configure on a bunch of internal machines. It also just works, and in the kernel (on Linux and modern OpenBSD), which means we don't have to worry about it breaking or about necessary daemons dying for one reason or another.

(So far all of this is me thinking about things. We'll probably keep on opening holes in our external firewall until this becomes impossible for some external machine, perhaps because it has to keep changing IP addresses.)


Comments on this page:

By Michael at 2022-09-28 02:17:03:

Have you ever considered Tailscale [https://tailscale.com/] as a solution for this?

By Joe at 2022-09-28 04:17:49:

Surely there's a third option?

Allocate address space in your network that routes to your wireguard 'server'. It doesn't need to NAT?

It also doesn't need to be a spof, there're are a couple of ways you can offer multiple endpoints externally. Internally, you've got simple solutions like keepalived?

By cks at 2022-09-28 18:41:52:

Tailscale is out of our budget (which is zero for something like this), and as far as I know requires running additional daemons on the machines. One of the appeals of bare WireGuard is that it's daemon-free; you configure it and you're done, with the kernel handling everything else.

We certainly could create a special inside subnet for the WireGuard IPs of external hosts and expose that, but it would require routing updates on all of our servers to add the new subnet and its gateway (which is, admittedly, more or less automated). The attraction of other options is that they're more contained (and as a side effect, also limit (internal) access to the WireGuard IPs of external machines, just in case they're running services we don't want internally accessible).

By Juan at 2022-09-29 22:49:53:

Ah, I had a good number of leads as to how to approach implementing a wireguard mesh network utilizing an IPv6. The internal network IPs would be set by truncating the last 64 bits from the public key and using them as the host bits. I got busy before I tried to figured out how to have kernel wireguard alert and message that a new client had connected.

Regardless, have you looked at headscale or the other wireguard mesh tools?

I find myself wondering if WireGuard could do something like IPsec Transport Mode. That way the IPs stay the same.

You can do IPsec without any daemons running via static transformation configurations.

Written on 27 September 2022.
« The lsb_release program and the /etc/os-release file
How I've set up my libvirt based virtual machines (in late 2022) »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Sep 27 21:09:04 2022
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.