2022-09-27
Reaching past our firewalls with WireGuard (some thoughts)
Our network design carries with it the implicit assumption that all of our machines (and all machines run by other people) are within our network perimeter, so we can safely expose dangerous services to them, like an unauthenticated SMTP 'smarthost' and a central syslog server. In the beginning this was completely true, but over time we've acquired a few machines that are outside our network perimeter (and if there is cloud in our future, we'll get more). We'd rather like for these external machines to have some access to our standard services; at a minimum we want them to be able to email us when they have issues liked failed disks, and it would be nice to be able to collect logs and so on. Our current solution to this is to poke holes through our firewall, but recently I've been tempted by the idea of using WireGuard as a more secure approach.
The appeal of WireGuard for this is that it's a lightweight service that requires little configuration or operation, and is now supported across all of our Ubuntu fleet. This creates two obvious options, depending on how much work we want to do on these external machines. The first option is to run WireGuard in a non-routed, "point to point" mode on each of the internal machines that have services we want to provide access to. The internal machine would expose its service(s) on an private WireGuard network as well as its normal IP address (in many cases this requires no service changes), and external machines would reach the service by talking to the internal machine's private WireGuard IP address. The one drawback to this is that it requires configuring each external machine to use the appropriate magic WireGuard IPs for these services, instead of the hostnames we normally use.
The other approach would be to create a NAT'ing WireGuard gateway machine and then configure external machines to route traffic to specific machines that run relevant services (our mail smarthost, our central syslog server, etc) through their WireGuard tunnel to the gateway. Because of the problem of network tunnels and asymmetric routing this mostly requires these internal machines to never want to initiate connections to the external ones (which is true for services today), and it makes the WireGuard NAT gateway machine a central point of failure for all external machines talking to all internal services. But it avoids configuring WireGuard on a bunch of internal machines and changing the service configuration on the external machines (since they can keep on using the regular hostnames for things like our SMTP smarthost).
(Not having to change the configuration of things on external hosts makes life much easier, because we don't have to keep two copies of configurations.)
You can do this (in either approach) with any VPN or secure IP tunneling system. The advantage of WireGuard is that it's very easy and lightweight, so it's simple to configure on the external machines and viable to configure on a bunch of internal machines. It also just works, and in the kernel (on Linux and modern OpenBSD), which means we don't have to worry about it breaking or about necessary daemons dying for one reason or another.
(So far all of this is me thinking about things. We'll probably keep on opening holes in our external firewall until this becomes impossible for some external machine, perhaps because it has to keep changing IP addresses.)