Wandering Thoughts archives

The yum versionlock problem

Update: I'm wrong about this. See the comments for more details.

Since I alluded to this issue recently, here is the problematic interaction between yum's versionlock plugin, which is used to 'pin' a particular version of a package so that it won't get upgraded, and yum's installonly feature, which is used by Red Hat and Fedora to keep only some number of kernels (by default 2).

Here is the problem sequence:

• you are running version X of the kernel without problems.
• Fedora releases version Y, so you yum update to it.
• when you reboot into version Y, you discover that it has a problem (for example, it reliably panics on your hardware, or your Ethernet card performance goes way down).
• you reboot back into version X, which still works.
• you want to keep version X around until Fedora releases a new kernel that works, so you list version X of the kernel in /etc/yum/pluginconf.d/versionlock.list to 'pin' it.
• Fedora releases a new kernel, version Z, and you 'yum update' to it.

What you expect to happen during the update is that yum will apply the 'only 2' default kernel policy but notice that version X is pinned and so either keep three versions around (two regular ones plus one pinned version) or remove kernel version Y. Instead yum removes your pinned version X and leaves you with only Y and Z (both of which may be broken).

Technically speaking this is not a bug as such, because the versionlock plugin does not promise to prevent a package's removal, it just protects it from being updated by newer versions; the kernel you locked isn't being updated, it's just being removed as a side effect of updating another package. But I do think it is both surprising and the wrong thing for yum to do, and that versionlock should prevent a package from being removed for any reason.

There are two workarounds. First, you can increase the number of copies of the kernel that are kept around (and remember to keep increasing it if kernel updates come out that don't fix the problem that keeps you at version X). Second, you can remove every version Y that doesn't fix the problem so that you only have one kernel version (version X) sitting around, although you'll have to not versionlock it if you want to update to new kernel versions to try them out.

How packaging systems should handle kernel updates

As a system administrator, I have some relatively strong opinions on how Linux distributions and packaging systems should handle kernel updates. Since I have just gone through the experience of yet another mass kernel update of our servers, I feel like writing them up:

• you should be able to have multiple kernels installed at once.
• it should be easy to tell which package's kernel you are running, and thus whether or not you are running the most current kernel.
• installing a kernel update should never overwrite an existing kernel image; it should always install a new kernel.

  (The one time when it is marginally acceptable to overwrite an existing kernel image is when the package update has literally no changes in the kernel; all it does is fix some packaging mistake. But that should be vanishingly rare anyways.)

• old kernels should get removed sooner or later, at a rate that is configurable but that has a sensible default. I do not need every kernel ever released for my distribution sitting around on my disks (especially if they have security holes).

  (Ideally there would be some sort of time-based minimum expiry, so I can say 'never remove a kernel until I haven't run it for a month'.)

• you should be able to 'pin' any particular kernel so that it is never removed. You really want this if you have a 'last known good' kernel and you want to keep experimenting with kernel updates to see if your distribution got it fixed this time around.

• the running kernel and its modules should never, ever get removed (or overwritten).

I don't know of any distribution that gets all of these right. Ubuntu fails spectacularly at never overwriting existing kernels; Red Hat has no good way to pin a particular kernel version so that it won't get removed (unless the yum versionlock plugin has very recently fixed its interaction with the 'keep the most recent N' plugin). No one has good kernel expiry.

Oh, and I prefer as much of this as possible to be in the kernel packages themselves, instead of in tools that are used to manage them, so that I cannot accidentally cause problems by using the wrong package management tool. This means that I somewhat prefer the Debian/Ubuntu approach of making different kernel versions have different package names to the Red Hat approach of making different kernel versions be different versions of the same package and putting the smarts in yum and other tools.

One consequence of Linux's dynamic network device naming

I knew that Linux did dynamic network device naming, but one consequence of that only really sunk in recently. Dynamic naming is relatively predictable for interfaces that are always on (especially when various distributions work hard to make them very persistent), but where it can really surprise you (and your software) is when the interfaces are dynamic. Like, say, PPP interfaces.

Just like Ethernet interfaces, the first active PPP connection gets ppp0, the second one gets ppp1, and so on. But unlike Ethernets, PPP connections can routinely come and go, and as a result it may not be clear to you (or your software) what PPP device is what connection. Also, it is easier to commit the old mistake of trying to bring up a connection that's configured as ppp1 when you don't have a ppp0.

(Scripts run by pppd itself get told the right network device, which can be both useful and potentially confusing.)

Unfortunately it looks like you cannot rename PPP devices, because it seems that you can only rename network devices that aren't active and when a PPP device isn't active, it doesn't exist at all. So I can't have pppdialup0 and pppdsl0 instead of, well, whatever pppN name they get right now.

One can half-fix this with software; pppd has a concept of a logical link name, and contrary to what the manual page appears to claim, it does pass this information to the ip-up script as the environment variable $LINKNAME. So your local link-up script can save the mapping between actual PPP device (the first argument) and the logical link name somewhere, and then other things can read it.

What some fdisk options actually do

Since I was just looking into this recently, I will write down what I've learned about the less obvious effects of some of the fdisk commands.

The u command

As a side effect of switching to using sectors as the size unit, fdisk stops rounding up the end of your partitions to the nearest track or cylinder boundary.

The c command

This stops fdisk from rounding up the start of partitions to the nearest track or cylinder boundary. (Fdisk gives this the cryptic description of turning off "dos compatibility".)

An example may help show the effects:

• with the default settings, a disk with just an extended partition that takes up the whole disk will have it start at sector 63 and end somewhat before the end of the disk.
• with u alone, the partition will start at sector 63 but run to the end of the disk.
• with u plus c, the partition will start at sector 1 and run to the end of the disk.

If you care about the exact size of your partitions, for example because you are trying to get some number of exactly identically sized partitions, you should use u. Using c is optional but will get you somewhat more disk space.

(Note that even with sector units and no DOS compatibility fdisk will still silently add one sector to your logical partition sizes, apparently so that partitions always start on an even sector number. This is not visible in fdisk output (which always reports the size of partitions in Kbytes, regardless of the setting of u); you have to use a tool like sfdisk to see the exact details.)

I see no reason not to use u and c all the time, with the possible exception of leaving some disk space at the front of disks for the bootloader. The tracks and cylinders that fdisk talks about are purely imaginary constructs on any disk made since at least the turn of the century, all of which have much more complicated internal geometries and actively hide them from you anyways, so you might as well just use straight sectors (or kilobytes, megabytes, and so on) and dispense with all the complexity and artificial limitations.