Fedora has significantly fumbled DKMS handling for Linux kernel modules

May 19, 2021

Recently I blamed DKMS for building one of my kernel modules for the wrong kernel. It turns out that this is not quite what happened, and in fact what happened is Fedora's fault. I found this out when I did yet another unsuccessful kernel upgrade attempt on my work machine, had the module problem happen again, and paid close attention to what was actually in my module directory (partly because this time around I was sure I'd checked for the problem before I'd rebooted).

As part of their customized kmod package, Fedora ships a program, /usr/sbin/weak-modules, which apparently comes from Red Hat Enterprise and which exists to, well, let me quote the comment at the start of the shell script:

weak-modules - determine which modules are kABI compatible with installed kernels and set up the symlinks in /lib/*/weak-updates.

This is the best documentation we have for the program because Fedora has not provided a manual page. Well, actually the current DKMS manual page says something about this, and what it says is so incredible that I'm going to quote it almost in full, with emphasis mine. This is from the section where it talks about various parameters you can set in a module's dkms.conf:

The NO_WEAK_MODULES parameter prevents dkms from creating a symlink into the weak-updates directory, which is the default on Red Hat derivatives. The weak modules facility was designed to eliminate the need to rebuild kernel modules when kernel upgrades occur and relies on the symbols within the kABI.

Fedora does not guaranteed a stable kABI so it should be disabled in the specific module override by setting it to "yes". [...]

Let me translate that for you: Fedora requires all DKMS modules to disable this feature that they themselves have added.

What happens if you use a DKMS-built module that doesn't do this is that Fedora's weak-modules script may well decide that the version of the module DKMS just built fresh for your newly-installed kernel is compatible with every previous kernel you have. It will then install symlinks from /lib/modules/<old-kernel>/weak-updates/<module>.ko to the very latest module you built, and then the Fedora version of depmod, modprobe, and so on will prefer those over the properly built copy in /lib/modules/<old-kernel>/extras/<module>.ko, and you will be unhappy because your module is not working.

This failure is a fundamental decision Fedora has made with its kernel builds. Kernel modules have a piece of information called 'vermagic' (you can see it with modinfo), which is the kernel version and broad environment the kernel module was built for. Fedora will not load a kernel module with a mismatched vermagic, which means that no kernel module can ever be (kABI) compatible across kernel versions.

Fedora shipping the weak-modules script is a sham. It cannot ever work as designed and intended, the weak-modules kABI check is broken on Fedora, and the only net result is that all DKMS modules on Fedora must add a special Fedora-only parameter in order to work right. If they don't, things work (only) as long as you never revert back to a previous kernel.

I would file a bug report but I cannot imagine that Fedora would accept it. They already know that this feature doesn't work; it's right there in the DKMS manpage. It's there anyway because, well, I don't know. This is the most user-hostile decision I think I've ever seen Fedora make.

(You might wonder how ZFS on Linux isn't affected by this. The answer is that they specifically turn it off. The dkms.conf for my out of tree it87 module doesn't.)

Written on 19 May 2021.
« Speculating on why DKMS and other Linux things are large shell scripts
The temptation to start using some Python type hints »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed May 19 23:48:19 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.