When Linux's rp_filter might make sense
I wrote a grumpy entry about net.ipv4.conf.*.rp_filter setting back here, where I said that it didn't make any sense. Well, I can actually come up with one situation where it may make sense: virtualization and thus virtualization networks.
One relatively common virtual machine setup is NAT-based, where the virtual machines get IP addresses on a private virtual network on the host. While the host doesn't route to its virtual network (or networks, if you have a big enough virtualization setup), it may be listening for various guest related services on its internal IP address. You don't want these services to be reachable from outside the machine; instead, you really do want the machine's private virtual network to be an isolated network. Using the rp_filter setting is the easy way to achieve this, and its drawbacks are irrelevant because the isolated network is both private and disconnected from any other real machine.
(Well. Most of the drawbacks. Interesting things may happen if your guest virtual machines try to talk to the real IP address of your host.)
While you can use ipfilters or policy based routing to achieve the same effects, both require you to know the IP addresses of some or everything involved; this means you need to generate the rules for your machine and update them when the machine's IP addresses or connectivity change. Using rp_filter is a lazy blunt hammer that gets you away from this.
However, I still believe that it's a mistake to default rp_filter to on. At least right now, I think that most users are not going to do virtualization and so the better approach (although it takes slightly more work) is to turn rp_filter on only when people configure virtualization.
(The counter-argument to this is that having rp_filter on all the time prevents the situation where you turn on virtualization and suddenly your networking explodes. Of course it does this by making your networking explode from the start, but you're more likely to notice that.)