Wandering Thoughts archives

2007-04-13

A little gotcha about binding services to interfaces

I ran into this over this about a year ago, but I got reminded of it today:

Regardless of what interface a service is bound to, its outgoing packets are going out through the interface that the default route sits on.

(Or the route to the specific destination.)

The only packets that are guaranteed to go out the same interface they came in on are those from sources on that interface's local networks.

This would have bit me hard a year ago if we'd had more load on the mirrors I set up. After the dust had settled, I noticed that while the mirrors listened on different subnets, the two machines involved actually had default routes that ran over the same 100 mbit subnet so their outgoing packets (which would have eaten all the bandwidth) would have been choked down to a combined 100 mbit/s.

The same thing is true of merely connecting to services by a specific IP. Your packets to the service will go through the right network, but the return packets may come out through a different one if the target machine is multi-homed. (This is true even if the target machine does not have IP forwarding enabled; IP forwarding only affects what the machine does with other people's packets.)

Getting around this generally requires policy based routing to route packets based on the origin IP as well as the destination. This is generally more complicated to set up than it needs to be, since what you really want is an option to make source addresses glued to interfaces; a glued source address would require that any packet coming from it must be routed out its interface.

(If there is no such route, you drop the packet with 'network unreachable'.)

sysadmin/RoutingVersusInterfaces written at 23:24:08; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.