Chris's Wiki :: blog/sysadmin/DisablingAccountChallenge

Disabling an account can be kind of complex

July 26, 2011

Question 31 on Tom Limoncelli's sysadmin test is:

G.31: Can a user's account be disabled on all systems in 1 hour?

This may be more complex than you think. I'll use our local environment as an illustration.

We have a central password distribution system. It'd take me about thirty seconds to edit the master /etc/shadow in order to lock someone's password in it (and then a couple of minutes for this to get propagated around). Some people would confidently stop here, and for many situations and many users it's actually good enough.

(I know that there's some Linux command that will lock the password for me, but it would honestly take me longer to figure out what it was and look up the option I need than it does to edit /etc/shadow directly.)

But if a user has some form of passwordless ssh set up, they can still log in even with a locked password. So I'd better remember to set the user's shell to an administrative shell that just prints a 'this account is suspended' message. (Do you have such a shell ready to go?)

But wait. Samba doesn't use the Unix password file; it has its own password file and we have magic set up to update the Samba password store at the same time as we update the normal Unix passwords. Simply editing the Unix /etc/shadow does nothing at all to the Samba password file, so a user with a locked password could still get access through Samba (some of our users might not even notice that they couldn't log in to our Unix machines). The simplest fix for this is to not lock the user's password in /etc/shadow but to use passwd to scramble it to a random value so that the Samba password store is updated too.

However, this isn't good enough, since there are a number of ways for users to run commands without actually logging in. Leaving these intact probably doesn't count as really disabling the account, especially if you're concerned that the user might be malicious and so might have left themselves back doors. In our environment, a user could have a crontab file or at jobs on any number of machines, they could have a pipe to a command in their .forward (or in a simple mailing list that they own), and they could even have CGIs on our web server (or worse, an entire user-managed web server). They might also have running processes on any number of our machines. To fix this I'd have to check every machine and decommission or kill anything that I found.

After all of this, I've managed to thoroughly disable a user's account on our Unix systems. But users have additional access in our environment in several ways, and if we're being thorough I need to get rid of them as well. First off, users may have VPN accounts; these are completely separate from their regular Unix account (with a different password). A disabled user should probably lose VPN access, so I'd better remember to edit that password file too.

Next, users may have registered specific machines on our general access network for laptops and other user-run machines. We probably don't want a disabled user to be able to get any network access through our networks, so I'd have to look up all of their machines in the DHCP database and remove or disable the entries.

(Can you find all of the machines registered to a user and remove their access? What about your wireless environment; how much access does the user have by still knowing your SSID and WEP or WPA wireless key?)

I could probably do all of this in an hour (ignoring delays in how long some things take to propagate around). But I'd certainly want to think about it carefully to make sure that I wasn't missing something in the corner, and if I was doing this in a hurry and under pressure I'd probably have missed something.

(Reversibly suspending an account is harder, assuming that you need to be this thorough. Deleting an account entirely instead of just disabling it is probably easier, because you don't have to worry about a lot of these things if you remove the user's /etc/passwd entry and delete their files.)

It's also worth noting that the idea of 'disabling' an account is imprecise. For example, should a disabled account still be able to receive email, or should we start bouncing email to it (and if so, should we use a special error message)? Should a disabled account's personal web page still be visible? The answers may depend on just why an account is being disabled and in any case these are policy issues, not technical ones.

(However, they interact with the technical issues. If a disabled account should continue to receive email, what do you do if the account's normal mail handling involves a piped program in the user's .forward?)

Comments on this page:

From 69.158.17.191 at 2011-07-26 18:49:33:

Any particular reason you don't like NIS and/or LDAP?

Nowadays NIS is fairly obvious ('ypcat passwd'), but LDAP is very handy, and allows one to also consolidate the Samba back-end as well.

By cks at 2011-07-27 01:05:21:

The answer for NIS is now in WhyNotNISToday. The answer for LDAP is more involved, but part of it is in OurPasswordPropagation; LDAP adds another server that has to be up in order for anyone to do anything.

(We are not going to put an LDAP server on the fileservers. They're Solaris and we're not masochistic or crazy. Even if we did, it would still be adding a daemon that has to be up and working right to the environment.)

From 87.79.236.202 at 2011-07-27 06:24:24:

Isn’t that exactly Tom’s point, though?

(Note: I want to make a purely formal argument here. I do think that you missed his point; whether you should accept his premises is another question. They seem sensible to me, but I am no sysadmin, much less an experienced one, so I have no firm opinion.)

As I understand the point, it is that whatever mechanism you use to mediate authorisation, there should be a single point from which it all can be controlled.

That does not mean that you need to do it via LDAP or NIS; or other service; or any form of centralised authentication. It can be some kind of replication or propagation just as well, or any other kind of federated auth. Whatever you pick is an implementation detail, and a red herring.

In other words, you shouldn't have to meticulously reconstruct all of the workings of your system every you need to shut off someone’s access. You should encode that knowledge somewhere in such a way that a machine can process it, because computers do not forget and do not slip.

(The argument reminds me of the difference between having a bunch of test scripts whose output you can pore over vs. an automated test suite that runs the same tests and compares them to expectations and gives you just a go/no-go result. The former can be made equivalent to the latter in terms of confidence it can provide, but it is far less liberating because it burdens humans with the sort of detail-orientation that computers are made for. In testing (any kind of it), we know that this is true. Thus I suspect his argument is sound.)

What your entry seems to say then, is simply that you have the sort of system Tom is warning against: that you fail this question on his test.

—Aristotle Pagaltzis

By cks at 2011-07-27 08:52:21:

I wasn't clear enough in my entry, because I agree with you. Our environment certainly looks like it passes Tom's test here, and easily at that. But it doesn't really pass the spirit of the test, because when you start looking at it 'disable an account' requires a lot more than 'lock the password'; in fact 'lock the password' turns out to be the wrong answer entirely. This show how sysadmins need to think about this carefully (ideally in advance), instead of just confidently going 'yes, we pass, we can lock the password in our central system'.

By Dan.Astoorian at 2011-07-27 10:07:18:

But if a user has some form of passwordless ssh set up, they can still log in even with a locked password. So I'd better remember to set the user's shell to an administrative shell that just prints a 'this account is suspended' message.

This is what we do. It isn't necessarily good enough either.

Yesterday I suspended a student's account for using SSH port forwarding to run BitTorrent, in order to get around the bandwidth limits imposed by the student residences. However, the student was still able to connect via SSH and do port forwarding, because "ssh -N -L[...]" does not require keeping any command running on the server side. The shell merely needs to exist (I don't think it even has to be in /etc/shells).

In my case, the fix for this was to add the user to the "denyssh" group we keep in /etc/group, which is listed in a DenyGroups statement in sshd_config for just such an emergency.

On some systems, locking the account by putting the appropriate string "*LK*" or "!!") into the shadow field will prevent ssh from allowing the user access even when password authentication is not in use; setting the account as expired (e.g., via usermod -e) may also do this. However, I don't know how reliable either of these should be considered--it may depend on many things, such as whether sshd is configured to use PAM or not.

--Dan

From 216.254.58.194 at 2011-07-27 14:41:37:

Not being snarky, but account locking is pretty stone simple: `passwd -l USERNAME` under Linux.

One of the first things I did when I came on board was to teach the Juniors how to do that so that there wouldn't be concurrent access issues on the centralized shadow, group, and passwd files anymore.

Running a local master server for this stuff (NIS/LDAP) in your network segment means you don't HAVE to synchronize hundreds or thousands of systems either, also reducing access issues. While the heart of system administration is certainly to automate routine tasks, elimination of the need for such is even better.

And sure, NIS is a bit outdated. But the last time ours broke was in 2007 and once before in 2004.

By Dan.Astoorian at 2011-07-27 15:45:27:

Not being snarky, but account locking is pretty stone simple: `passwd -l USERNAME` under Linux.

Which puts a leading "!!" into the /etc/shadow entry (see my earlier comment)... Which may or may not prevent access via methods other than password authentication (e.g., RSA or DSA keys, or hostbased authentication), depending on whether PAM is being used, how it is configured, sshd compile-time options, and possibly other factors as well.

On my RHEL5 systems, for instance, passwd -l USERNAME doesn't prevent ssh access via RSA keys (contrary to what sshd(8) says, and arguably this makes it a bug), but expiring the account via usermod -e does.

So, no, account locking is not as stone simple as it should be.

--Dan

From 97.107.130.220 at 2011-07-28 03:12:24:

This shows how sysadmins need to think about this carefully (ideally in advance)

Ah! I see.

—Aristotle Pagaltzis

Written on 26 July 2011.

«	The risks of using CentOS are split

Why not YP, er, NIS