First, the background. net.ipv4.conf.*.rp_filter controls some IP address source validation filtering done on incoming IPv4 packets. It has three values:

(A more formal description is in ip-sysctl.txt in the kernel documentation. Like all interface sysctls, it can be set separately for each interface, as a default, and for all interfaces.)

I don't understand how this can possibly work. Well, I understand how it works, I just don't understand how it can possibly do any good in most configurations. And I don't understand how a setting of '1' can possibly work at all in multihomed configurations where the multihomed machine is not the sole router for every network it's connected to that is not where its default route points.

First, as far as I can tell a setting of '2' is equivalent to '0' if you have a default route set (the usual case). With a default route set, all source IPs are reachable and so '2' will never discard packets, which is exactly the same as '0'.

For a machine with a single network interface and a default route, all settings are equivalent (for the same reason as above; all source IPs are reachable through your single interface). If you do not have a default route, either '1' or '2' will discard packets that come from networks you do not have routes for.

It is the multihomed case where things explode. Suppose that you have a multihomed host with two network interfaces, net-1 and net-2, with IP-1 on net-1 and IP-2 on net-2. With an rp_filter value of 1, a machine on net-2 cannot talk to this machine's IP-1 address unless the packets pass through the multihomed machine on the way to net-1, ie the multihomed machine is the router for the net-2 machine. If the packets go through another router, they will arrive on the multihomed machine's net-1 interface but the replies would go out the net-2 interface, so they fail the check.

Effectively this creates a bad version of an isolated interface, with the packet reachability restrictions but without the multiple split routing tables that make multihomed hosts actually work. As a bonus it hides the restriction deep in the networking sysctls, where you have to be an expert to find it.

(I suppose that there are some advantages to this half-hearted approach, in that it avoids some limits in the policy based routing version of it.)

By the way, I stumbled over this courtesy of Ubuntu 10.04 setting rp_filter to 1 by default. We have multihomed non-routing machines, and when we set up an Ubuntu 10.04 test version things promptly exploded. If I was not already suspicious of network sysctls, we could have spent quite a lot of time trying to find out just why the machine was ignoring certain sorts of network traffic.

(As it was I did 'sysctl -a | fgrep net. | sort' on both a 10.04 and an 8.04 machine and then looked for settings that were different. Ubuntu 10.04 may not be the first version that sets this, but 8.04 definitely didn't.)

PS: a much more useful version of this sysctl would be a 'private' flag on interfaces. If an interface had the private flag set, packets with a source IP address that was routed through that interface would only be accepted on that interface; all other interfaces would discard such packets.