Wandering Thoughts archives

The risks of forcing frequent password changes

People can talk about the risks of not changing passwords at the drop of a hat, but they rarely talk about the risks of forcing people to go through frequent password changes. This matters a lot, because there are significant risks.

(And when people do acknowledge the risks, often it is to decry or mock the 'bad habits' that people adopt.)

The obvious problems are that people can't remember things at the drop of a hat just because you want them to, and people have trouble coming up with good passwords (especially at the drop of a hat). So people write down their new password, which is much more likely to be a weak password, because it's much easier to come up with (and remember) weak passwords than strong random ones.

But the direct, obvious problems pale next to the big one: frequent password changes train people not to take security seriously. Being made to change your password frequently is so obviously security theatre (especially if it's treated as a big deal) that it undermines everything else; when you engage in one obvious bit of security theatre, people are predisposed to assume that everything else is also security theatre.

(Remember that most people don't understand security (and probably aren't interested in doing so), and so they have to trust your judgement. But if they wind up feeling that your judgement is clearly flawed on one bit of security practice, this inevitably lessens their trust in all of your judgements.)

This leads me to a suggestion: if you are forced by higher powers to implement mandatory password changes, at least acknowledge to your users that it is a stupid idea (and ideally work with them to make it as painless as possible, for example by teaching them how to come up with one strong password that they can then vary back and forth).

(Yes, I'm aware that this may be politically infeasible.)

Frequent password changes as security mythology

One of the things I see trotted out in response to potential security issues is a suggestion that we make users change their password frequently. In addition to all of the practical reasons not to do this, it's useful to ask one of the most important question in security: what risks does this protect against, and how big are they?

The concise answer is that it doesn't actually protect against any fundamental risks. What it does is limit the damage done to your systems by compromised passwords (if you are very lucky, it contains them to 'none'). And if your passwords are not getting compromised, it does nothing (except annoy users).

In other words: if your passwords are getting compromised, forcing frequent password changes is not actually solving the problem. Your real problem is that your passwords are getting compromised, and that's what you need to fix. Forcing password changes is merely a mitigation strategy, not a fundamental cure, and worse I think that it is generally an ineffective mitigation strategy.

(This feels like a terribly obvious insight now that I have actually written it out.)

Sidebar: what risks password changes protects against

Okay, I exaggerated, but not very much.

Let's start with a simpler question: what effects does someone changing their password have? (Besides causing them to write the new password down on a postit note and, if you are lucky, stick it besides their money or credit cards.)

Answer: it cuts off access to anyone else who knows their old password.

There's several risks that this protects against:

• that their old password is compromised and hasn't been exploited yet.
• that their old password is compromised and is being exploited, but the exploitation hasn't been detected yet.
• that their old password will be compromised in the future using information that the attacker has already captured (the classical 'steal the hashed passwords and brute force them later' issue).

So, how big are these risks, especially when compared to the very real risks that forcing frequently password changes creates? My personal opinion is that all of these three risks are pretty low, because passwords are compromised rarely and when they are compromised they seem to be used almost immediately, usually in obvious ways.

Another problem with doing your own sysadmin automation

In addition to the stuff in AutomationCosts, there's another problem (or cost) with custom sysadmin tools: they're almost always at least not-good software, and sometimes they're outright bad. They pretty much can't help but be because of how they're created.

The typical custom tool is written by at most a small group of people (who are often not actively practicing programmers), is not looked over by very many people, and on top of that is often written in a bad language for programming. I'm not saying that you have to be an active programmer to write good programs, but I think that it helps; the odds are stacked against a small team that doesn't do much programming producing something good. Similarly, having lots of people poke at your work helps improve its quality, especially if they are outside people.

(Yes, there are sysadmins who are good, active programmers, but generally the people who really like programming become programmers, not sysadmins.)

In short, your custom tools just don't get the degree of attention and involvement that vendor software or open source software gets, which stacks the odds against them being as good as such software.

The usual rejoinder to this depressing view is that you'll generalize your custom tools and then share them for the world to adopt and improve, which will get you all the benefits of open source software in general. Unfortunately this does not seem to work in practice; the LISA conference proceedings are littered with tools that have gone basically nowhere since being announced. You might be an exception, but history suggests that the odds are still stacked against you.

The costs of doing your own system administration automation

There is a constant tension in system administration between doing things by hand in the standard way, versus creating your own custom systems to automate the work. Doing things by hand takes longer and is more tedious, so it sounds like automation should be a slamdunk, but the problem with automation is that you've created a new local piece of software that has to be updated, maintained, learned by other people (especially new people), and understood.

(Or to put it another way, you have taken something that is someone else's problem and made it your own problem.)

Note that the people who originally wrote the systems are going to be bad at assessing how obscure and hard to learn and so on those systems are, because of course the custom systems make perfect sense to them (or at least they're very likely to). To really evaluate this, you need an outsider, such as a new person coming into your group; if they start wanting to strangle you after a while, you have a problem.

This has been a hard lesson for me to learn, partly because I am naturally inclined to be an enthusiastic tool builder and partly because I have spent a great deal of my career as (effectively) the sole sysadmin of an environment. I owe a debt to my co-workers for determinedly prying my eyes open.

(You get a similar 'sole sysadmin' effect if people have very distinct areas of responsibility, where eg one person is the mailer person and no one else really tries to keep up.)

This doesn't mean that you shouldn't ever automate; it just means that you should bear this in mind and make sure to ask yourself if the extra complexity is worth it. Sometimes the answer should be no.

Why people don't automate sysadmin stuff

Yesterday's entry raises an obvious question: why don't people automate things ahead of time, before they have so many systems that they have to automate or die?

The simple answer is that it is easier for them not to automate than it is for them to automate. But why is it that way?

There are at least two reasons:

• there is pretty much no standard way to automate things at the various levels, especially across different Unixes and platforms. This means that you have to build an automation setup yourself, or at least evaluate a bunch of open source options to see which ones actually work in your environment.

• when there are vendor tools, the vendors have generally done a bad job of making them available and convenient. The area that this particularly manifests in is that vendors have done a bad job of letting you grow into using the tools, where you can do things like install a system once by hand and then have it adopted by the vendor's auto-install framework.

From my perspective, part of the problem is that vendors often aim too high. Instead of adopting a Unix tools approach, where they try to create relatively simple pieces that can be used as building blocks, they try to create an 'Enterprise Management Application' or the like that meets all the needs in one go.

(The lack of standard tools has other costs, but that's another entry.)

Or in short: right now, the lack of standard tools and good integration means that it takes a bunch of work to automate, enough that it's hard to justify if you're not running enough systems.

Automation changes as systems grow

Over the time I've worked in system administration, I've observed a common pattern that I will summarize this way:

As your system count grows, you wind up automating the automation.

Here is the levels that I've seen people (myself included) go through:

• managing systems by hand, one by one.
• using tools that let you run a command on all of your systems or classes of your systems.
• automating things for clusters, so that you don't have to explicitly run commands on each system.

There are well-managed places that climb up these levels early, but in many cases it happens because you have no choice; you have enough systems that you can't afford to manage them with any lower level.

(I haven't personally worked in environments that went beyond the third level, but I imagine that when you grow bigger you can't afford to create clusters by hand so you wind up automating the cluster formation and configuration.)

What made this strikingly visible to me was moving backwards in the levels, when I went from an environment where there were enough systems that I was at the third level to an environment where there were few enough systems that they're mostly being managed at the first and second level. It's been somewhat of a culture shock.