A realization about our VPN and IPv6 traffic

June 21, 2021

At work, we operate a VPN for our users. The VPN is used to access both internal resources inside our networks and university resources that are normally only available from 'on-campus' IP addresses. Because of the latter, and for historical reasons, our VPN servers are configured to tell VPN clients to route all of their traffic through the VPN, regardless of the destination. In other words, the VPN makes itself the default route for traffic. Today, in the process of investigating an unfortunate Google decision, I realized that there's an important qualification on that statement.

(We actually support two different sorts of VPNs, OpenVPN and L2TP, and have two servers for each type, but all of this is a technical detail. Conceptually, we have 'a VPN service'.)

We and our networks are IPv4 only; we haven't even started to implement IPv6, and it will probably be years before we do. Naturally this means that our VPN is IPv4 only, so its default route only applies to IPv4 traffic, which means that all of the client's IPv6 traffic bypasses our VPN. All of the IPv4 traffic flows through the VPN, but if your client has a working local IPv6 connection, any IPv6 traffic will go through it.

The first consequence of this is for traffic to places outside the university. An increasing number of ISP networks provide IPv6 addresses to people's devices, many of those devices prefer IPv6 where possible, and an increasing number of sites are reachable over IPv6. Connections from people's devices to those sites don't go through our VPN. But if you move the same device over to a network that only provides it an IPv4 address, suddenly you're going through our VPN to reach all of those sites. This makes troubleshooting apparent VPN based connection problems much more exciting than before; we may have to disable IPv6 during our tests, and we may have to find out if a user who's having problems has an IPv6 connection.

The second consequence is that some day some of the university's on-campus websites may start to have IPv6 addresses themselves. Traffic to these websites from IPv6 capable clients that are connected to the VPN will mysteriously (to people) be seen as outside traffic by those on-campus websites, because it's coming directly from the outside client over IPv6 instead of indirectly through our VPN over IPv4. There are also some external websites that have historically given special permissions to the university's IPs. If these websites are IPv6 enabled and your client is IPv6 enabled, they're going to see you as a non-university connection even with the VPN up.

There probably isn't anything we can sensibly do about this. I think it would be a bad idea to try to have our VPN servers grab all client IPv6 traffic and block it, even if that's possible. Among other things, there are probably IPv6 only ISPs out there that this would likely interact very badly with.

(Our VPN isn't officially documented as a privacy aid for general Internet usage, although people may well use it as that. So I don't consider it a security issue that the current configuration leaks people's real IPv6 addresses to sites.)


Comments on this page:

By Gareth Eason at 2021-06-22 02:28:07:

Have you considered biting the bullet and supporting IPv6? It's been around and carrying internet traffic for over 20 years now, and we have run out of IPv4 addresses some time back, making IPv4 only problematic, and encouraging inappropriate NAT and breaking the TCP/IP end-to-end principle.

Wireguard is efficient, secure, supports IPv4 and IPv6, and has clients for a wire range of operating systems and platforms. Tailscale and other platforms provide a reasonable and improving management interface for large-scale wireguard deployments, making it a little easier to manage, and bringing you into the world of IPv6

By cks at 2021-06-22 11:17:06:

Supporting IPv6 requires significant changes to our environment and probably to many systems in it, and will not be possible in some parts. There are a huge number of unsettled issues, such as IPv6 reverse DNS, how Android devices don't do DHCP6, and ripple effects anywhere we deal with 'IP' addresses. We are sure to find software issues with programs, too. And there's not very much benefit to us for adding IPv6.

It would be nice to have IPv6. But it's very hard to justify what would likely be person-months of effort to design and deploy it into our environment today, plus significant risks of things going wrong.

(And then there is the issue of other people having problems with their IPv6 connectivity and support. Enabling IPv6 means that our users will start using IPv6 to talk to those other people and run into those problems. From the user's perspective, we did something and suddenly they have problems talking to some outside site.)

By cks at 2021-06-22 11:25:26:

On the Wireguard issue: I'm a big fan of Wireguard and use it myself, but deploying Wireguard to regular people quite clearly needs something like Tailscale that automates the key provisioning. I can just imagine the havoc if we tried to have a self-service portal that offered either 'generate your keys and paste your public key here' or 'here is your private key, copy it to your Wireguard configuration'.

(Using Tailscale itself would unfortunately require a significant amount of money based on their pricing; we would need at least subnet routing, and we have a lot of users. This would be a very hard sell to the administration, since our current VPNs are free.)

By Opk at 2021-06-23 01:52:43:

My employer's VPN failing to mess with IPv6 has been really useful to me while working from home. The work laptop can still see my printer and I can do all my work from my own desktop, mostly only using the laptop as an ssh jump host.

I fear they'll close this hole at some point and I'll need to be creative to find a workaround. IPv6 has the advantage of not needing me to do anything that administrators might take a dim view of in terms of compromising security. There's a trend towards increasingly locking down provided work tools which can be frustrating when it makes it harder to work the way I want.

By Arnaud Gomes at 2021-06-23 05:10:44:

You're probably overly pessimistic about the state of IPv6 implementations. There have been no significant software issues for the last 10 to 15 years in my experience, and nowadays pretty much all network operators can push packets around correctly.

The reverse DNS issue is real, but I'm not sure it is really so important in practice. In my experience it mostly matters for email, and then it matters less and less as more and more mail hosters insist on DMARC and DKIM anyways. Plus, you're talking about VPN clients here, and you can use your VPN infrastructure itself to manage client IP addresses (at least you can with OpenVPN, I don't know about L2TP), things are much easier here than in SLAAC-land.

Oh, and do yourself a favour, forget DHCPv6 even exists. It has no real-life use except subnet delegation, and you probably don't want to do this dynamically at your scale.

   -- A

With regards to IPv6 for the purposes of this situation, a "full" roll-out is probably not needed. It would probably be enough to get an IPv6 block allocation, configure your routers for it to reach the outside world, and assign addresses to the VPN server to to serve out.

You could then roll-out IPv6 to the rest of the network as needed/desired sometime in the future.

By cks at 2021-06-23 09:21:31:

The situation with DHCP6 nicely illustrates the practical problem with adding IPv6, because we use DHCP as a core element of our network access control for some internal networks. In addition to the direct access control, there are any number of networks where we don't want machines to show up on random IP addresses and so we'll never enable SLAAC on them; machines will either do DHCP6 and automatically get various useful things set up right or their owners will have to manage setting static IP assignments and everything else on their own. The latter will be unpopular.

From 193.219.181.219 at 2021-06-24 02:17:45:

Supporting IPv6 requires significant changes to our environment and probably to many systems in it, and will not be possible in some parts. There are a huge number of unsettled issues, such as IPv6 reverse DNS, how Android devices don't do DHCP6, and ripple effects anywhere we deal with 'IP' addresses. We are sure to find software issues with programs, too. And there's not very much benefit to us for adding IPv6.

It would be nice to have IPv6. But it's very hard to justify what would likely be person-months of effort to design and deploy it into our environment today, plus significant risks of things going wrong.

I agree that you're probably overly pessimistic. An IPv6 rollout does not have to be all-or-nothing. It is perfectly fine to configure it first on a few servers (or your home WG tunnel), then on your VPN gateway, and just stop there.

As long as you don't make some service IPv6-only, dual-stack isn't going to affect the IPv4-only parts of the network. Any IPv4-only client software wouldn't be requesting IPv6 addresses in the first place, and likewise an IPv4-only server wouldn't have AAAA records in DNS.

(I've slowly enabled IPv6 with SLAAC on all wired LANs, but Wi-Fi in most areas still remains IPv4-only – for reasons related to unmanaged switches carrying tagged VLANs – and the Wi-Fi clients carry on using IPv4 like before.

I've also had to remove several IPv6 AAAA records from DNS after discovering problems with appliances having broken v6 stacks, or Happy Eyeballs interfering with switches that were tying management sessions to an IP address, or our mail system mishandling SPF for IPv6 senders... nothing really that would neccessitate turning it off entirely though.)


Android devices don't do DHCP6

As for DHCPv6, do you need it on VLANs which have Android devices? I assume it's to be able to track which addresses belong to whom, but I've found that monitoring ARP and ND is a more flexible way of doing so (specifically, scraping routers' neighbour caches every few minutes, though I've seen another university use libpcap to collect it live).

This doesn't rely on any particular addressing scheme or configuration method, and is even able to track addresses that someone configures statically (as those will still show up in the ARP cache), so its usefulness and reliability well outweighs scraping being somewhat ugly in principle.

Written on 21 June 2021.
« Some notes on building Firefox from source on Ubuntu
I like WireGuard partly because it doesn't have 'sessions' »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jun 21 22:46:17 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.