Dual identity routing with Linux's policy based routing

In the last entry I talked about the idea that an end node machine in a VPN actually has two distinct identities that need to be routed separately: the inside identity that should be routed over the VPN, and the outside identity that should be routed over the Internet. In this case each identity has an IP address associated with it; call the inside identity's IP address I and the outside identity's IP address O.

We can do this sort of routing with Linux's policy based routing, which fortunately is not as scary as it looks. To simplify somewhat, Linux does policy based routing by having multiple routing tables, and then providing rules to pick between them. We need two new routing tables, one for each identity, but fortunately we don't need them to have many routes; all they actually need is a default route.

So we set up the tables like this:

ip route add default dev VPN table 10
ip route add default dev REAL table 11

(Here VPN and REAL are your VPN and REAL network interfaces; you may need to add 'via GATEWAY' to the one for your real connection. My setup uses PPPoE DSL, so I can just throw my packets in the direction of ppp0 and have everything work right.)

Now we need two policy rules so that traffic explicitly from one or another of our identities is sent out the appropriate interface:

ip rule add from I iif lo priority 5000 table 10
ip rule add from O iif lo priority 5001 table 11

(The table numbers and the priorities are mostly arbitrary.)

This says that traffic explicitly from I should consult table 10 before theoretically falling through to the default main routing table. Similarly for traffic explicitly from O, which gets sent to table 11. Since tables 10 and 11 have a default route, we never actually fall through.

The final thing to do is to set up which networks default to using the inside or the outside identity when we connect to them. We do this in the regular routing table by choosing which interface we send a route to, for example to route 128.100.5.0/24 over my VPN connection I do:

ip route add 128.100.5.0/24 dev VPN

(Unless you change your default route, the default identity is your outside identity and thus you just need to add route entries for stuff you want to go over the VPN. Because the most specific route matches, you can make some large network go through the VPN, and then exempt specific subnets if you need to.)

This works because both the VPN and the REAL interfaces have local IP addresses associated with them, which determines the local IP address for outgoing connections that haven't explicitly specified it. (Connections that have specified the local IP address will get stolen by our explicit rules before we reach the main routing table.)

As a bonus you can make outgoing connections use a specific identity by specifying the origin address; for example, I could still connect to a machine in 128.100.5/24 as my outside identity by using 'ssh -b O <whatever>'. (This might be necessary if the VPN connection dies for some reason.)

(I owe a real debt to the documentation for the ip command, which turned out to be quite understandable once I actually sat down and read it carefully. There's a lot of useful and interesting stuff there.)