Wandering Thoughts archives

Why Red Hat 7.3 is (still) so present on servers

Red Hat 7.3 came out in the summer of 2002, but much to various people's horror you can still find it on servers (along with Red Hat 7.2, another perennial favorite). I think that there are at least two reasons for this, and they work together in combination.

First, Red Hat 7.3 was the last Red Hat release until Fedora Core 2 where you could run a stock kernel. From Red Hat 8 through FC 1, you needed a 2.4 kernel with the NPTL patches, which meant that you had to use a Red Hat kernel, and in turn this probably forced people into upgrading machines running those later distributions as kernel security issues turned up. On RH 7.3, you could just build your own kernel in response to problems.

(Our remaining 7.3 machines are not running Red Hat kernels, for example; they're running relatively current 2.4 kernels we built ourselves.)

As part of this, RH 8 and RH 9 were at least seen as much more experimental distributions than RH 7.3. Red Hat 7.3 enjoyed a reputation of stability and quality which made it a good choice for several years, partly because it was seen as a point release in the 7.x series instead of a new 'major' version.

(For example, our 7.3 machines were installed in mid to late 2003, but it was clearly still seen as a good choice even then, despite Red Hat 9 having been released.)

Second, I think that Red Hat 7.3 just arrived at the right time, when people were increasingly willing to install Linux and Linux was looking increasingly like a good choice. And systems often don't get upgraded if they keep on working, so the first serious wave of adoption can stick around for quite a while.

(The 2.4 kernel series had a lot of good stuff going for it by 7.3, including stability, decent NFS, good SMP support, and ext3. I can't remember if it had XFS, but I believe SGI was at least working on XFS at the time, which served as a useful endorsement.)

(This entry was sparked by Dave Jones' entry about running into obsolete Red Hat kernels.)

chkconfig --add considered misleading

Presented in the traditional illustrated form:

# chkconfig ntpd || echo not running
not running
# chkconfig --add ntpd
# chkconfig ntpd || echo not running
not running

I like Red Hat's chkconfig, but its --add option is so badly misnamed it winds up irritating me half the time I use it. One indication of how bad it is is that the chkconfig manpage needs to use a paragraph to half explain it, and the explanation is exceedingly technical.

Part of the problem is that chkconfig is overloaded: it is used by sysadmins to enable and disable services, and it is used by RPMs to add and remove init.d scripts. Because --add half works to enable services (it enables anything that defaults to being enabled), it is tempting to use it all the time by reflex, and then get irritated when it doesn't work.

(The theoretical correct way to enable a service is 'chkconfig <service> on'. Notice how this is a completely different method of operation from chkconfig --add <service>, so you have to remember not just a different command-line option but a different syntax.)

Another chkconfig problem is that right now, init.d scripts overload the 'what runlevels this should be enabled in' field to say whether or not they should default to being enabled. This means that a service that defaults to being disabled can't say what runlevels are appropriate for it, with the result that to be correct you cannot just 'chkconfig <service> on', because that may enable it in inappropriate runlevels.

(The common case is enabling network services in runlevel 2. Of course, I doubt anyone really uses runlevel 2 so in practice this doesn't really matter. But it irritates me, because sometimes I'm a nit-picker.)

Why Evolution is not my favorite mail reader

I will admit up front that I am not an IMAP person; I'm an MH and exmh person and have been for years. The only reason I'm playing around with IMAP is that I'm more or less responsible for an IMAP server, so it behooves me to be able to test whether or not it's working. I picked Evolution as my IMAP test environment for various reasons including that it's the Gnome default.

As a result of this, I have formed some opinions about why Evolution is not my favorite mail reader:

• it crashes, and too often. I'm an old-fashioned person; I expect my mail reader to just work, and not to explode every so often.
• to add insult to injury, the bug reporting system it's hooked up to in Fedora Core goes all the way through quizzing you and getting you to fill in information, only to announce that Evolution bugs aren't managed through the system it files bug reports with.

• getting Evolution to notice IMAP folders created outside of itself seems to take multiple rounds of quitting and restarting Evolution. I guess I can be thankful that I have a fast machine.

• Evolution desperately needs an 'accept this SSL certificate permanently' option. Otherwise, if you are using a self-signed certificate, you get prompted about it every time you start Evolution up.

• there appears to be no way to export your filters from one Evolution and import them into another. Apparently it is a freakish thing to want to read email from work and home with the same filtering settings.

  (It turns out that the magic way is to copy the filters.xml file from one install to another; it's found in .evolution/mail. You probably need to do this while the target Evolution install is not running.)

• filters mysteriously fail some of the time, or mysteriously fail to have some options (in my case, 'move to folder') work. As far as I can tell, there is no way to debug this sort of issue; Evolution is totally silent about the operations of filtering, what matches, and so on.

In fact, Evolution is silent pretty much any time something doesn't work, which makes it very frustrating whenever something goes wrong. Which it seems to for me with moderately distressing regularity.

(For example, Evolution managed to get one IMAP folder into a state where it could half-see that there were messages there (sometimes), but totally refused to show me any of them. Fixing this was impossible, and eventually something just erased all of the messages.)

I also have to say that it is annoying that no two IMAP clients seem to be able to agree about all of the fine details of how to deal with messages, which makes it hard to switch between clients for testing. Given that this is a client/server protocol to start with, it would be nice if the clients were more interoperable.

In summary, I'm really glad that I'm just using Evolution and IMAP for testing and I don't have to use it as my real mail environment.

(While bits of this have been accumulating for a while, this grump has been brought to you today by the process of trying to bring up a second IMAP server running different IMAP software on a different OS platform, and thus attempting to make sure that people can switch back and forth relatively transparently.)

A limitation of Debian's /etc/network/interfaces control file

Unless I am missing something, in Debian and Ubuntu there is no way to bring up an interface without having it try to obtain an IP address in some way. The interfaces(5) manpage suggests that 'auto ethN', possibly with 'iface ethN inet manual' should work, but it doesn't seem to go in my testing on Ubuntu.

You might rationally ask what use there is in bringing up an interface without an IP address. It turns out that there's a number of things that such up but unconfigured interfaces are either good for or necessary for (depending on your perspective), including:

• running tcpdump and similar packet capture programs to do traffic accounting and/or monitoring.
• bringing up your DSL PPPoE connection, since PPPoE uses the Ethernet purely as a transport.
• bridging virtual guest machines onto an Ethernet (or VLAN) that the host machine is logically not on (although it clearly has physical access to them).

The Ubuntu machine I ran into this on was our traffic accounting system, and we definitely never want the monitoring interface to be assigned an IP address, or even respond to packets. (If we were being thorough, this means that we should turn off some normal Linux network settings so that the interface never responds to rogue ARPs for the machine's management IP address.)

The workaround we're currently using is to set 'iface ethN inet dhcp'. There's no DHCP server on the monitoring segment that will ever respond to the accounting machine's DHCP request; in fact, I believe that there is basically no untagged traffic flowing over that network segment.

(And yes, if this ever changes we could get a peculiar surprise.)

Sidebar: setting up unconfigured but active interfaces in Fedora

To set up such an interface in Fedora (and probably Red Hat Enterprise), you want the obvious minimal ifcfg-ethN control file:

DEVICE=ethN
ONBOOT=yes

Put this in /etc/sysconfig/network-scripts and you're good to go.

Why indirect xdm probably doesn't work on your Linux machine

We recently had a fun adventure of debugging indirect XDMCP on our Ubuntu LTS 6.06 (Dapper Drake) servers. The problem was that you couldn't go from the chooser on one server to logging in on another server, although you could log in to the machine the chooser was coming from.

It turns out that this is broken by default in pretty much every modern Linux distribution, and the culprit is IPv6 support. In fact it makes a good illustration of why adding IPv6 support is not an easy slamdunk. The bug goes like this:

• when you pick a machine in the chooser, the chooser server sends the target machine a UDP packet saying basically 'I am forwarding you a request to manage display Y on IP address X'.

• xdm prefers to bind to the IPv6 version of the XDMCP UDP port, when it can.

• the code that receives and processes forwarded requests insists that the address of the machine it is being asked to manage is the same length as addresses of the address family that the listening socket is for.

• IP addresses are not the same length as IPv6 addresses, so when xdm receives a request to manage an IP address over its IPv6 UDP socket, it discards it as an error. (Cleverly, it discards it without so much as a debugging message to complain about the problem.)

Reading the source code, it is easy to see how the bug was introduced. Originally the code dealt with only IP(v4) and Unix domain sockets; in this context it was sensible for the IP socket to insist that it was being asked to manage a 4-byte address, not something that looked more like a Unix domain socket address. When IPv6 support was introduced, it was almost certainly done by just copy and pasting the IPv4 code and changing the IPv4 specific things to IPv6 things.

(The code itself is cut & paste identical, but I can't be sure how it came about.)

This illustrates one of the problems of the would-be IPv6 migration: you can't just mechanically add IPv6 support to programs, you have to look through them to pick up more subtle implications of the whole change. In the old days of xdm, it made no sense to get a Unix domain socket address through an IP forwarded request or vice versa; in the new era, it does make sense to get an IPv4 address through an IPv6 forwarded request (and it may make sense the other way around, too; you'd have to think about it). This is not just adding support mechanically, this is a logic change.

(The bug is still there in xdm 1.1.4 and the current development version available through gitweb.freedesktop.org . Interested parties can look at the routine forward_respond in xdmcp.c.)

Sidebar: how to work around this

There are two ways to work around this:

• make xdm listen for XDMCP packets only on the IPv4 INADRR_ANY, not the IPv6 one. You do this by adding the line

  LISTEN 0.0.0.0

  to /etc/X11/xdm/Xaccess, which controls various bits of XDMCP.

• turn off IPv6 entirely. You can only do this if IPv6 was built as a module in your kernel, but it probably was. You want to blacklist the module net-pf-10; you can probably do this by adding a file to /etc/modprobe.d that contains:

  alias net-pf-10 off

Unless you're one of a rare band of people, you aren't losing anything by turning IPv6 off entirely except some future aggravation with the next program that has subtly buggy IPv6 support.