I am generally fond of OpenBSD's PF packet filter but every so often I run across a seemingly arbitrary limitation that drives me up the wall. Today's limitation is on where you can redirect packets to as part of NAT'ing and general address translation. I'll start by sketching out a simplified version of the problem I'm trying to solve.

Part of our complex networking setup is a scheme where specific internal machines, sitting on 'sandbox' subnets in private address space, can be reached by the outside world through public IP addresses that sit on what is effectively a virtual subnet. Through a complex dance involving two firewalls, these machines are bidirectionally NAT'd to their public IPs when they talk to the outside world. Our problem is that sometimes internal machines try to use the public IPs, and we'd like to make that work. What we want to do is conceptually simple: when a packet from the internal network and to the public IP shows up on the sandbox firewall, it should be rewritten to the internal IP instead and put back on the internal network. Something like, in pf-ese:

pass in quick on $int_if from <int_lan> to $PUBIP rdr-to $INTIP route-to $int_if

(It's not necessary to rewrite the source address and in fact it's a feature to not do so. Update: as covered in comments, it may be necessary to rewrite the source address to force return traffic to flow through the firewall to be fixed up.)

As it happens, OpenBSD PF is specifically documented (in the pf.conf manpage) to not allow this:

Redirections cannot reflect packets back through the interface they arrive on, they can only be redirected to hosts connected to different interfaces or to the firewall itself.

In the fine OpenBSD tradition this is in fact not completely true. The specific LAN segment that is $int_if actually has two separate subnets on it for historical reasons and machines on the other subnet can talk to $INTIP through this rdr-to rule without problems. It's only machines on the same subnet that can't (and not because PF blocks the packets; I've checked).

What I assume is happening is that PF and OpenBSD's routing stack are interacting badly. Under normal circumstances a router will not route a packet from host A on a subnet to host B on the same subnet (at most it will send an ICMP redirect). In an ideal world PF would be able to bypass this restriction when it rdr-to's something, especially with an explicit route-to (in my books, route-to should mean 'shut up and send the packet out that interface no matter what'). In this world PF apparently can't, which is an irritating limitation that gets in the way of what I maintain is a perfectly sensible thing to want to do.

(There are any number of cases where you might want to redirect traffic nominally to the outside world back to an internal machine.)

PS: as the pf.conf manpage notes, theoretically the way around this is to add NAT'ing with a pass out rule. I was unable to get this to work when I tried it but I might have been using options that were slightly wrong. I assume that this NAT'ing process is enough to fool the routing system into accepting the packet as something that could be validly routed.

(On the other hand, if 'pass out' is applied after routing is done I don't see how this can work. It would make sense for it to be a post-routing action, since routing is what normally decides the outbound interface, but the pf.conf manpage doesn't document whether this is the case or whether some deep magic is happening.)