Wandering Thoughts archives

Linux iptables compared to OpenBSD PF (through a real example)

One of the things I've been asked about in response to our attachment to OpenBSD PF is how OpenBSD PF differs from the Linux alternatives, especially iptables. I don't have a good, satisfying answer to that, so today I'm going to cheat by showing a realistic case written out in both and then discussing some of the less obvious differences between the two.

To implement our custom NFS mount authorization, we need to block access to a collection of NFS-related ports (for both TCP and UDP) unless the source machine is either on our own server subnet or is in a dynamically maintained table of machines that have authenticated to us through our special system. In OpenBSD PF syntax, this looks something like the following:

table <NFSAUTHED> persist

pass in quick on $IFACE inet proto { tcp, udp } \
    from { 128.100.3.0/24, <NFSAUTHED> } to any \
    ports { 111, 2049, 10100 }

block in quick log on $IFACE inet proto { tcp, udp } \
    from any to any \
    ports { 111, 2049, 10100 }

(For obvious reasons I haven't actually tested this, although I think it would work. There is a cultural assumption embedded here in the form of $IFACE; it's usual, at least here, to have pf.conf know what the system's interfaces are called.)

What I usually say about Linux iptables is that it's an assembly language for creating firewalls. As the equivalent of an assembly language it's very flexible, but it's also rather verbose and there are almost always a bunch of different ways to do something. With that said, here's the actual iptables ruleset we're using (and I know it works). Because we're using ipsets, our ruleset has to be represented as a series of commands (as far as I know), since we have to run commands to create the ipsets before we create the iptables rules using them.

ipset create nfsports bitmap:port range 0-12000 counters
ipset add nfsports 111
ipset add nfsports 2049
ipset add nfsports 10100

ipset create nfsauthed hash:ip counters

# Accept from localhost as a precaution
iptables -A INPUT -p tcp -i lo -m set --match-set nfsports dst -j ACCEPT
iptables -A INPUT -p udp -i lo -m set --match-set nfsports dst -j ACCEPT

# Accept our server network.
iptables -A INPUT -p tcp -s 128.100.3.0/24 -m set --match-set nfsports -j ACCEPT
iptables -A INPUT -p udp -s 128.100.3.0/24 -m set --match-set nfsports -j ACCEPT

# Accept authorized machines.
iptables -A INPUT -p tcp -m set --match-set nfsports dst -m set --match-set nfsauthed -j ACCEPT
iptables -A INPUT -p udp -m set --match-set nfsports dst -m set --match-set nfsauthed -j ACCEPT

# Reject everyone else, with logging
iptables -A INPUT -p tcp -m set --match-set nfsports dst -j NFLOG --nflog-prefix "deny"
iptables -A INPUT -p udp -m set --match-set nfsports dst -j NFLOG --nflog-prefix "deny"
iptables -A INPUT -p tcp -m set --match-set nfsports dst -j REJECT
iptables -A INPUT -p udp -m set --match-set nfsports dst -j REJECT

(The choice of explicitly allowing loopback versus guarding everything else with '-i $IFACE' is partly cultural and partly pragmatic; we would have to write the latter in a lot more places than on OpenBSD, and interface names are often rather less predictable.)

Some of the rule count difference here is illusory. For example, if we applied the OpenBSD pf.conf stanza on an OpenBSD machine and then dumped the actual resulting rules with 'pfctl -s rules', we'd discover that pfctl had expanded each of those '{ ... }' groups of things out into separate rules, one for each option. If I'm doing the math right, that means our two lines of pf.conf would turn into 24 actual rules, which is more than the Linux version has. However, generally what matters is how many rules people need to write, not what the rules expand to when implemented at a low level. Here OpenBSD PF gives us a number of tools to write compact rules that set out everything we're doing in one place (which is important for coming back later and understanding what your rules are doing).

(OpenBSD tables specifically only contain IP addresses and networks, so as far as I know we can't create an equivalent of the nfsports ipset we're matching port numbers against. This does mean that in OpenBSD, we couldn't change the ports we were matching against on the fly; if they changed, we'd have to update the pf.conf rules and reload them.)

A larger difference is that these rules don't actually mean quite the same thing, because Linux iptables are normally stateless while OpenBSD is stateful by default and in customary use. Here this actually would make a difference in how we want to operate the overall authentication system. In Linux removing something from the nfsauthed ipset immediately removes all its access, because the rules are checked for every packet, while in OpenBSD we would also have to kill off the removed IP's state table entries (if any) with 'pfctl -k', because once a TCP connection is in the OpenBSD state tables it completely bypasses pf.conf rules.

Our OpenBSD 'block in quick ...' rule is also not necessarily natural in PF because of another cultural difference. OpenBSD PF configurations are normally written with denying packets being the default, with a default catch all 'block in log all' somewhere in your pf.conf; in a default-deny environment, a specific block rule such as this is unnecessary. But with Linux, you normally leave iptables accepting packages by default on an otherwise un-firewalled machine such as we have here, which means that we have to write out those four lines of 'log and drop' iptables stuff at the end. In a real OpenBSD PF configuration this would probably cause us to write more pf.conf rules to specify what other inbound traffic we wanted to allow.

PS: Linux ipsets are now well over five years old so they're pretty universally available in distribution versions you actually want to run, but in the past you could easily find Linux systems that didn't have them available (they weren't in RHEL/CentOS 5, for example). Without ipsets, this setup would be far more difficult. OpenBSD PF has basically always had tables.