We have been progressively replacing a number of old one-off Linux machines with up to date replacements that run Ubuntu and so are based on our standard Ubuntu install. One of those machines has a special feature where a group of people are allowed to use passworded sudo to gain access to a common holding account. After we deployed the updated machine, these people got in touch with us to report that something had gone wrong with the sudo system. This was weird to me, because I'd made sure to faithfully replicate the old system's sudo customizations to the new one. When I did some testing, things got weirder; I discovered that sudo was demanding the root password instead of my password. This was definitely not how things were supposed to work for this sudo access (especially since the people with sudo access don't know the root password for the machine).

Whether or not sudo does this is controlled by the setting of 'rootpw' in sudoers or one of the files it includes (at least with Ubuntu's standard sudo.conf). The stock Ubuntu sudoers doesn't set 'rootpw', and of course this machine's sudoers customizations didn't set them either. But when I looked around, I discovered that we had long ago set up an /etc/sudoers.d customization file to set 'rootpw' and made it part of our standard Ubuntu install. When I rebuilt this machine based on our standard Ubuntu setup, the standard install stuff had installed this sudo customization. Since we'd long ago completely forgotten about its existence, I hadn't remembered it while customizing the machine to its new purpose, so it had stayed.

(We don't normally use passworded sudo, and we definitely want access to root to require someone to know the special root password, not just the password to a sysadmin's account.)

There are probably a lot of things that we've added to our standard install over the years that are like this sudo customization. They exist to make things work (or not work), and as long as they keep quietly doing their jobs it's very easy to forget them and their effects. Then we do something exceptional on a machine and they crop up, whether it's preventing sudo from working like we want it to or almost giving us a recursive syslog server.

(I don't have any particular lesson to draw from this, except that it's surprisingly difficult to de-customize a machine. One might think the answer is to set up the machine from scratch outside our standard install framework, but the reality is that there's a lot from the standard framework that we still want on such machines. Even with issues like this, it's probably easier to install them normally and then fix the issues than do a completely stock Ubuntu server install.)