How not to set up IP aliases on Ubuntu (and probably Debian)

April 6, 2010

Suppose that you need some IP aliases on an Ubuntu machine. So you go to /etc/network/interfaces and slavishly make yourself some, copying the main stanza a number of times to make entries that looks like this:

auto eth0:0
iface eth0:0 inet static
  address 128.100.1.A

(repeat for every additional IP alias, increasing the number and replacing A with B and so on for all of the different IP aliases.)

What's wrong here is the additional gateway statements for each IP alias; you do not want to specify gateways for IP aliases. The problem with all of these gateway statements is that they create multiple default routes:

$ ip route list | fgrep default
default via dev eth0 src 128.100.1.A metric 100
default via dev eth0 src 128.100.1.B metric 100
default via dev eth0 src 128.100.1.C metric 100
default via dev eth0 metric 100

(You have to use 'ip route list' to see this; 'nestat -nr' will tell you that you have multiple default routes but not how they differ.)

These routes differ only in that three of the four specify that the local IP address is something besides the machine's primary IP address (the 'src <IP>' bit).

When you have multiple default routes with the same metric, Linux picks which one to use semi-randomly (and it will change which one it uses from time to time). Since different default routes come with different local IP addresses, your outgoing connections (and UDP requests) will periodically come from a different IP address. This is comedy gold, especially when combined with a cautiously configured firewall that hasn't been configured to pass outbound traffic from some (but not all) of those IP addresses.

Troubleshooting this is part of where the comedy gold comes in; things will work sometimes and not at other times, with the problem coming and going randomly (in reality it comes and goes as the machine chooses different default routes to use, with different local IP addresses). You can have a 'telnet outside-host port' command fail and then your TCP-based traceroute succeed and look fine, for example.

(This happened to us on an Ubuntu 8.04 system. Since Ubuntu and Debian use basically the same system for handling network configuration, I suspect that it would also happen on a Debian machine. It may also happen in other distributions, depending on what they do when you give an IP alias a gateway.)

Written on 06 April 2010.
« An important thing about how ZFS handles spares
Our current mail system's configuration »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Apr 6 00:07:15 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.