Linux PAM leads to terrible error messages from things like
Here is a puzzle for you. Suppose that you're trying to change your password on a typical Linux system, as happens periodically (and as we make new logins on our systems do immediately), and you get the following:
; passwd Changing password for user cks. Current password: passwd: Authentication token manipulation error
What has gone wrong here? What should you do to fix it? Should you try again, or instead send email to your system administrators to get them to fix it?
Well, you don't know, because
passwd and Linux's implementation
of PAM have combined to
create a terrible error message through robot logic, where the error message is completely technically
logical and correct but useless in practice. The most likely cause
of this message is that you've mis-typed your current password,
but there are other possible causes if things have gone wrong in
the depths of PAM. The only people who can start to disentangle
this is your system administrators, or in general anyone who can
look at PAM's logs (normally in syslog), because only there will
you find extremely valuable and much more precise messages like:
passwd: pam_unix(passwd:chauthtok): authentication failure; logname= uid=19 euid=0 tty=pts/6 ruser= rhost= user=cks
Even this isn't really clear, but with sufficient painful experience
you can decode this to that the passwd command was verifying your
password through traditional Unix
/etc/shadow encrypted passwords,
and the password you typed didn't 'authenticate', ie didn't match
the encrypted password.
One of the reasons this is a terrible error message is because normal people have essentially no chance at all of understanding it (as I can assure you from our experience of supporting the people who use our systems). The best you can do is use a wrapper script that puts a big explanatory message around the whole thing, and even then people get confused.
(And if other things go wrong and the same message gets printed out, you're really confusing people; you've claimed that the problem is that they're using the wrong password, except they know that they're not. At least they'll probably email the system administrators at that point.)
I'm not sure if the PAM API provides any way for PAM modules such
as pam_unix to provide a more specific error message. This
particular error message is the generic PAM error string for
PAM_AUTHTOK_ERR, which is the equally generic PAM error code
that pam_unix is forced to return in this situation. You can
see the full list in the pam(3) manpage.
Keeping backup ZFS on Linux kernel modules around
I'm a long term user of ZFS on Linux and over pretty much all of the time I've used it, I've built it from the latest development version. Generally this means I update my ZoL build at the same time as I update my Fedora kernel, since a ZoL update requires a kernel reboot anyway. This is a little bit daring, of course, although the ZoL development version has generally been quite solid (and this way I get the latest features and improvements long before I otherwise would).
One of the things I do to make it less alarming is that I always keep backup copies of previous versions of ZFS on Linux, in the form of copies of the RPMs I install and update. Naturally I keep these backup copies in a non-ZFS filesystem, because I need to be able to get at them even if the new version of ZFS isn't working (possibly just with the new kernel, possibly in general). I haven't needed these backup copies very often, but on the rare occasions when I've had to revert, I was very glad that they were there.
(You don't always run into immediate failures to bring ZFS up; sometimes there are merely stability or other issues in a new development change, and you want to roll back to a previous one. In those cases it's okay to have the previous versions on a ZFS filesystem, because you can probably use ZFS enough to grab them.)
Not everyone uses development versions of ZFS on Linux, but I suggest that you keep backup copies of older versions even if you only use released ZoL versions. You never know when you may run into an issue and be glad that you have options.
(That I keep backup copies of previous versions and want to have them accessible outside of ZFS is one reason that I doubt I'll ever use ZFS on Linux on my root filesystem. System recovery is much easier in many scenarios if ZFS isn't required to at least boot the system, get it on the network, or access the root filesystem from a live CD.)
One obvious requirement here is that you should never update ZFS pool or filesystem features until you're absolutely sure that you'll never want to revert to a ZoL version that's too old to support those features. This generally makes me quite conservative about updating pool features; I want them to be in a ZoL release that's been out long enough to be considered fully stable.