Fedora has significantly fumbled DKMS handling for Linux kernel modules
Recently I blamed DKMS for building one of my kernel modules for the wrong kernel. It turns out that this is not quite what happened, and in fact what happened is Fedora's fault. I found this out when I did yet another unsuccessful kernel upgrade attempt on my work machine, had the module problem happen again, and paid close attention to what was actually in my module directory (partly because this time around I was sure I'd checked for the problem before I'd rebooted).
As part of their customized
kmod package, Fedora ships a program,
/usr/sbin/weak-modules, which apparently comes from Red Hat
Enterprise and which exists to, well, let me quote the comment at
the start of the shell script:
weak-modules - determine which modules are kABI compatible with installed kernels and set up the symlinks in /lib/*/weak-updates.
This is the best documentation we have for the program because Fedora has not provided a manual page. Well, actually the current DKMS manual page says something about this, and what it says is so incredible that I'm going to quote it almost in full, with emphasis mine. This is from the section where it talks about various parameters you can set in a module's dkms.conf:
NO_WEAK_MODULESparameter prevents dkms from creating a symlink into the weak-updates directory, which is the default on Red Hat derivatives. The weak modules facility was designed to eliminate the need to rebuild kernel modules when kernel upgrades occur and relies on the symbols within the kABI.
Fedora does not guaranteed a stable kABI so it should be disabled in the specific module override by setting it to "yes". [...]
Let me translate that for you: Fedora requires all DKMS modules to disable this feature that they themselves have added.
What happens if you use a DKMS-built module that doesn't do this
is that Fedora's
weak-modules script may well decide that the
version of the module DKMS just built fresh for your newly-installed
kernel is compatible with every previous kernel you have. It will
then install symlinks from
/lib/modules/<old-kernel>/weak-updates/<module>.ko to the very
latest module you built, and then the Fedora version of
modprobe, and so on will prefer those over the properly built
copy in /lib/modules/<old-kernel>/extras/<module>.ko, and you
will be unhappy because your module is not working.
This failure is a fundamental decision Fedora has made
with its kernel builds. Kernel modules have a piece of
information called 'vermagic' (you can see it with
modinfo), which is
the kernel version and broad environment the kernel module was built
for. Fedora will not load a kernel module with a mismatched vermagic,
which means that no kernel module can ever be (kABI) compatible across
Fedora shipping the
weak-modules script is a sham. It cannot ever
work as designed and intended, the
weak-modules kABI check is
broken on Fedora, and the only net result is that all DKMS modules
on Fedora must add a special Fedora-only parameter in order to work
right. If they don't, things work (only) as long as you never revert
back to a previous kernel.
I would file a bug report but I cannot imagine that Fedora would accept it. They already know that this feature doesn't work; it's right there in the DKMS manpage. It's there anyway because, well, I don't know. This is the most user-hostile decision I think I've ever seen Fedora make.
(You might wonder how ZFS on Linux isn't
affected by this. The answer is that they specifically turn it off.
dkms.conf for my out of tree
it87 module doesn't.)