Wandering Thoughts archives

A tip: Always include NAT in your Linux kernel configuration

Here's an important Linux kernel configuration tip: always compile NAT for your kernels, whether built in or as modules.

I often build my own kernels on machines (usually so I can use stock kernel.org kernels). Some of these machines are slow, and in general I like to leave out options I don't need. In the past, I have often looked at NAT and decided that I clearly wasn't going to need it. (After all, none of these machines were going ever to be firewalls.)

However, as I found out the hard way, there is another good reason to include NAT: local port redirection.

Local port redirection is a neat feature where you can map one server port to another. For example, if the idea of running Apache as root gives you the hives you can set it up to run as a normal user on port 8080, and transparently redirect port 80 to port 8080.

It turns out that transparent port redirection is useful for all sorts of tricks. And it's part of NAT; if you don't configure in NAT, you don't have it available.

So how do you do local port redirection once you have NAT? The full answer is in section 6 of the Linux NAT HOWTO, but the terse version is:

# iptables -t nat -A PREROUTING \
    -p tcp --dport 80 \
    -j REDIRECT --to-port 8080

(You may want to specify '-i eth0' or the like too. Or other conditions, to do redirection only for certain people. Want known spammers to see an entirely different SMTP server? You can do that.)

A smooth disk swap

It's nice when everything works like it's supposed to; the most excitement we had today was swapping in a replacement drive in this server. (I like Mondays for this sort of thing, since I'll be in the office for the next four days in case problems come up.)

The old IDE disk needed replacement because it was starting to remind us that it was consumer electronics by flaking out on seeks under heavy IO load every so often (hopefully it was the drive and not the cabling). Fortunately we are smart enough to use IDE disks in RAID-1 pairs so swapping drives is actually pretty easy, apart from the physical bits.

This server isn't ritzy enough to have pluggable drive bays; it's in an ordinary 4U rackmount case, although one that has been nicely engineered. As part of the good engineering, it has all of the drives in a 'drive cage' assembly; undo some screws, pull the cables off, and you can lift the entire thing out.

So we opened the case up, pulled the drive cage out, took it off to a desk (much more convenient than the machine room), swapped the old and the new drive (important safety tip: always label the old drive), and dropped everything back together. Elapsed time: about 25 minutes, mostly due to lots of screws.

Because the two disks are identical, and partitioned identically, putting the new disk into action was more or less like this:

# sfdisk -d /dev/hda | sfdisk /dev/hdc
# mkswap /dev/hdc2; swapon -a
# mkfs /dev/hdc1; mount /boot2
# cd /boot2
# dump -0f - /boot | restore -xf -
# mdadm -a /dev/md0 /dev/hdc5
# mdadm -a /dev/md1 /dev/hdc6
[... you can see how this goes ...]

Linux people may not have seen sfdisk before. I quite recommend it; it's a very convenient, rather Unixy command for fiddling around with partition tables. (The incantation used above copies /dev/hda's partitioning to /dev/hdc, which is exactly what we wanted.)

Tomorrow will be more thrilling, since backups of one partition of one server are now definitely failing consistently. Mysteriously.

The many consoles of Linux

People who are considering setting up serial consoles for their Linux machines often wind up a little bit confused about what's necessary and what sort of features are possible. A lot of this is natural, because the PC is fundamentally convinced that the console is the more or less mandatory video display.

This results in a situation where there are many different consoles in a Linux system. I'll inventory them, in the order that they get encountered during booting a system.

1. where the BIOS output appears. Expensive servers (and expensive addins) can change this from 'only the video display'
2. what the bootloader talks to; how to configure this varies with the bootloader.
3. where Linux kernel messages go. This is set by the kernel's 'console=...' arguments; you can specify multiple places. The kernel listen for so-called 'magic SysRq' keys on all of these.
4. what is /dev/console. This has to be a single real device, and is either the last console=... device listed or the video console.
5. where you can log in after the kernel has finished booting, which is determined by what /etc/inittab says to start getty on.

/dev/console is what init talks to during system startup (and system shutdown). This means that it's where startup messages appear (barring things like Red Hat's simplified graphical boot process) and what you need to interact with during single-user mode or if an automated fsck fails and needs a person to help it out.

In many situations, the ideal answer for 'what device is the console' is 'both the video display and a serial line', and most of these many consoles allow this. /dev/console is the unfortunate exception; you have to pick just one, because the Linux kernel requires it to map to only a single real device.

(I sympathize with the Linux kernel about /dev/console, honestly. The issues involved with merging simultaneous input from multiple real terminal devices are kind of tricky, especially if you want to avoid hooking into very low-level input routines down in the depths of drivers.)