How not to set up IP aliases on Ubuntu (and probably Debian)
Suppose that you need some IP aliases on an Ubuntu machine. So you go to
/etc/network/interfaces and slavishly make yourself some, copying the
main stanza a number of times to make entries that looks like this:
auto eth0:0 iface eth0:0 inet static address 128.100.1.A network 220.127.116.11 netmask 255.255.255.0 broadcast 18.104.22.168 gateway 22.214.171.124
(repeat for every additional IP alias, increasing the number and
B and so on for all of the different IP aliases.)
What's wrong here is the additional
gateway statements for each IP
alias; you do not want to specify gateways for IP aliases. The
problem with all of these
gateway statements is that they create
multiple default routes:
$ ip route list | fgrep default
default via 126.96.36.199 dev eth0 src 128.100.1.A metric 100
default via 188.8.131.52 dev eth0 src 128.100.1.B metric 100
default via 184.108.40.206 dev eth0 src 128.100.1.C metric 100
default via 220.127.116.11 dev eth0 metric 100
(You have to use '
ip route list' to see this; '
nestat -nr' will tell
you that you have multiple default routes but not how they differ.)
These routes differ only in that three of the four specify that the
local IP address is something besides the machine's primary IP address
src <IP>' bit).
When you have multiple default routes with the same metric, Linux picks which one to use semi-randomly (and it will change which one it uses from time to time). Since different default routes come with different local IP addresses, your outgoing connections (and UDP requests) will periodically come from a different IP address. This is comedy gold, especially when combined with a cautiously configured firewall that hasn't been configured to pass outbound traffic from some (but not all) of those IP addresses.
Troubleshooting this is part of where the comedy gold comes in; things
will work sometimes and not at other times, with the problem coming and
going randomly (in reality it comes and goes as the machine chooses
different default routes to use, with different local IP addresses).
You can have a '
telnet outside-host port' command fail and then your
traceroute succeed and look fine, for example.
(This happened to us on an Ubuntu 8.04 system. Since Ubuntu and Debian use basically the same system for handling network configuration, I suspect that it would also happen on a Debian machine. It may also happen in other distributions, depending on what they do when you give an IP alias a gateway.)