Third party Linux kernel modules should build against non-running kernels

May 7, 2012

For my sins, I have to deal with third-party kernel modules (some of them open source, some of them not so much). When you deal with third party modules, you get to rebuild them every time you update the kernel. At one level, almost everyone has this down pat these days; you run a command or two and you're done, with everything handled properly.

(I would like to say that this is a very welcome development. I've been around Linux long enough to remember when it was much more work and pain.)

At another level almost everyone gets this wrong, because they all force you to build their modules against the current running kernel and only the running kernel. This probably sounds fine for ordinary users but it makes sysadmins angry because it makes our lives more difficult. There are two problems.

First and obviously, it increases the service downtime. When you force us to build modules against the running kernel, the upgrade process is to install the new kernel, reboot the machine to activate it (which takes down the service, not just because of the reboot but because we're without your module), sit there rebuilding your modules with the service down, and then either activate the service or even reboot the machine again. If we could rebuild against a non-running version, we could install the new kernel, rebuild your modules, and then reboot; the only downtime would be the actual reboot.

Second, it makes kernel upgrades more dangerous. With third party modules there's always the chance that the module will not rebuild against your new kernel. If we can only rebuild against the running kernel, we have to reboot the machine (taking the service down) before we find out whether or not your modules will rebuild happily. If we can build against a non-running version, we can install the new kernel, try to rebuild, and if it fails we know we don't even have to schedule a downtime because there's no point.

(Yes, in theory everyone has test machines. This is still annoying, and sometimes you actually don't.)

I somewhat sympathize with the people building third party kernel modules; I'm sure that building against the running kernel simplifies your life and it's nice to be able to immediately test that your newly compiled module can be loaded. But don't make it mandatory and do provide an 'I know what I'm doing, just compile the thing against <X>' option. Sysadmins will thank you.

As a side note, being able to rebuild kernel modules in advance makes it possible to do opportunistic kernel upgrades. If we have to reboot the machine for some other reason, or even if the machine crashes and reboots on its own, we can switch it into a new kernel in the process. This is especially valuable for unattended reboots at odd hours, where there won't be a sysadmin there to rebuild the modules by hand at the time.

(Another case is situations with mass reboots, where you don't want to be babysitting individual machines as they come up in order to get them back into service.)

Comments on this page:

From at 2012-05-08 04:14:03:


My Nvidia Tesla powered machines always catch me out!

From at 2012-05-08 14:54:58:

I sympathize and get your point. But, in your opinion, why hasn't DKMS caught on enough to relieve (either partially or fully) this issue?

-Brad (

By cks at 2012-05-09 10:35:28:

I don't know why vendors haven't adopted DKMS, but clearly they haven't. Some of them may have overall build systems that predate DKMS and some of them may find that it's not installed by default on systems (it wasn't on my new Fedora 15 install).

(The vendor kernel modules I have experience with are VMWare's, NVidia's CUDA stuff, Digi's Etherlite drivers, and VirtualBox. I think VirtualBox can use DKMS if it's there, but it looks like it wasn't there by default on our Ubuntu install at the time.)

For IET, I suspect that DKMS doesn't match how the IET project wants to maintain and develop their kernel module. My impression is that the main code base targets a relatively current kernel; if you're using an older kernel they patch it on the fly with backwards compatibility hacks. Someone could take their work and repackage it for DKMS, but the project itself is focused on their own development.

Written on 07 May 2012.
« Why I wound up using Django
Things I will do differently in the next building power shutdown (part 2) »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon May 7 22:55:30 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.