Wandering Thoughts archives

Explaining an RPM oddity

Recently, Jordan Sissel tweeted:

Hey guys I beat the final boss of RPM: gist.github.com/4053631

Because I believe that gists may expire, I'm going to quote it here:

Two packages. Same name. Same file. Different epoch.

Both installed simultaneously.

rpm.

pork(~/projects/fpm) % rpm -ql fizz
/tmp/test
/tmp/test
pork(~/projects/fpm) % rpm -qa | grep fizz
fizz-1.0-1.x86_64
fizz-1.0-1.x86_64
pork(~/projects/fpm) % rpm -qa --qf '%{NAME} %{EPOCH}-%{VERSION}-%{RELEASE}\n' | grep fizz
fizz 0-1.0-1
fizz 1-1.0-1

There are several things going on at once here, but once you understand them all I think you'll see how this very peculiar situation comes about 'naturally' through general RPM processes.

First, RPM has always allowed you to install more than one version of an RPM at once provided that they didn't conflict; this was how RPM-based systems handled having several versions of the kernel installed at once. Back in the days of using rpm directly, you had to remember to install normal upgraded packages with 'rpm -U' instead of 'rpm -i'; doing the former would replace the old version while doing the latter would try to install the new version along side the old one, which generally didn't work too well.

In the beginning (I think), two RPMs conflicted if they both tried to supply the same file. At some point, this changed so that two RPMs only conflicted if they tried to supply the same file with different contents; if they supplied the same file with the same contents, you could install both at once. This was a core part in enabling multi-architecture support in RPM, which is what lets you install 32-bit and 64-bit x86 RPMs alongside each other. Of course these overlapping files normally happen because the 32-bit and 64-bit packages both have some common architecture-independent files like manpages or the like, but it's valid for any two RPMs (even two RPMs for the same architecture) to have common files.

(It's possible that RPM allowed some overlapping files very early in order to deal with badly done RPM packages that claimed to own common directories like /usr/bin or the like.)

Finally we get to RPM epoch numbers. The quick version is that epoch numbers are a hack to deal with upstream packages that either change their version numbering scheme or that don't have sensible ones in general. Because it's generally uninteresting (almost all packages have what is effectively an epoch of '0') and because it's not something that users should care about, RPM doesn't show it by default. However, it is formally part of the version number of an RPM and so two RPMs that differ only in the epoch have different version numbers and are different RPMs.

(This should normally not happen. If it does it means that the upstream reused version numbers, ie they did two separate and different '1.0' (or whatever) releases of the same package.)

So now we see what's going on here. We have two different version numbers for the same RPM package and the packages don't have any conflicting files (they both provide the same version of one file). So RPM will let you install both at once. Meanwhile its default output hides the epoch, making everything look mysterious.

A reminder: string concatenation really is string concatenation

Once upon a time when I was starting to write Python, I scribbled down the following code:

def warn(s):
  sys.stderr.write(sys.argv[0] + ": " + s + "\n")

(More or less. My actual code had an error and so didn't even work.)

Many Python programmers are wincing, because of course string concatenation is both somewhat inefficient and not the idiomatic way to do this; you should be using % string formatting. But there's another somewhat more subtle reason to avoid code like this, one that I ran into recently when I stumbled over this code the hard way by having it blow up in my face.

The surrounding code went something like this:

try:
  o, r = getopt.getopt(....)
except getopt.error, cause:
  warn(cause)
  ....

This failed. You see, the subtle problem with string concatenation is that it really is string concatenation. Unlike % formatting, it will not try to str() objects to convert them to strings; if they are not strings already, it just fails. It is of course easy to overlook this if you usually give your code actual strings; passing in a non-string object that can be stringified may be an uncommon corner case that you don't test explicitly.

This code actually exposes an interesting effect of Python's slow changes between Python 1.x and Python 2. Back in the old days exceptions actually were strings instead of objects that can be string-ified, and so this code could work when fed one of those exceptions. I wrote the program this code appears in back in 2003 or earlier and I believe we were still using Python 1.5 at the time (although it wasn't the current version even then); the 1.5.2 version of the getopt module appears to still have been using string exceptions at the time. So this might have been less crazy back then than it appears now (although it was still the wrong way to do it plus my actual implementation had a bug).