2022-04-28
The practical problem with /etc/pam.d on long-lived Linux systems
Yesterday I wrote about how a 20-second program startup delay turned out to be because of a stray line in an /etc/pam.d file. Unfortunately, things in /etc/pam.d are especially prone to this problem on systems that have been around for a long time and have been upgraded from version to version of their Linux. The short version of why is that there are too many things modifying /etc/pam.d files.
The ideal situation for /etc/pam.d files would be if the only thing that changed them was the packages that supplied them. Then they would be like program binaries; they would quietly and transparently get updated to new versions as part of package updates, including distribution version upgrades. A less ideal situation would be if the only two things changing /etc/pam.d files was the packages that supplied them and the system administrator. Pretty much every Linux package manager has features that are designed to deal with this, things like RPM's .rpmnew files and Debian's process for asking you what you want to do about the situation.
Unfortunately, /etc/pam.d files are historically modified by other programs as well under various circumstances; as they install themselves, when the system authentication configuration changes, and so on. Automatic modifications by programs of package-supplied files is generally a kiss of death for keeping them up to date. Even if the system administrator doesn't also make their own changes, package managers usually provide essentially no support for sorting out the situation, not even so much as a three way diff between the 'base' version (the old package version), the current version, and the package's new version.
(One reason for this omission is that it would require keeping around the original packaged version of all such files so that you can create the diff at all.)
Adding to the fun is that /etc/pam.d files are critical to your system working. If you break one, you may not be able to log in or use sudo (or su), or core infrastructure may stop working. This is by design (PAM fails closed, denying access if there's things wrong), but it makes any changes to a working environment an unusually high stakes activity. Add to this that PAM is tangled in general and it's no surprise that busy system administrators mostly don't touch their PAM stacks unless they have to. If the package update process didn't automatically handle things and it still works, don't touch anything.
It's possible that other programs modifying /etc/pam.d files has now gone out of style. I certainly hope it has, but I haven't looked.
(This problem of multiple programs automatically changing configuration files is one reason a great many configuration systems have moved to having a directory of configuration snippets. It's far easier to deal with that and to keep everything straight. /etc/pam.d files have yet to make that shift and to be fair, they present some unusual problems for it since you generally want very fine grained control over PAM ordering.)