An unchanging system should be stable

June 24, 2005

One of my principles of system administration is an unchanging system should be stable. Stable means more than 'doesn't crash', it means 'doesn't need to be fiddled with all the time'; no hand-holding, no cleaning up afterwards. It means a system that can be left to run quietly in the corner.

This does mean that you have to make it so that ordinary things can't cause the system to explode. The two big ones are making sure that system logs don't get too big and that temporary directories like /tmp and /usr/tmp get cleaned out regularly. (Fortunately modern systems mostly do this for you.)

After I've done this, I've found that things that keep demanding my attention are usually symptoms of some deeper issue that I need to deal with:

  • I've failed to automate a necessary procedure
  • I need to fix or work around some piece of buggy software
  • there's a dying or broken piece of hardware somewhere
  • the systems are underconfigured or overloaded

There is a rare and unfortunate final case:

  • a vendor's stuck me with braindead software that demands I interact with it by hand.

And really, if the systems aren't overloaded, the users aren't asking for changes, and there's no buggy software or broken hardware, why should a system need attention? Everything left looks a lot like monkey make-work, whether self-inflicted or forced on you by vendors.

Sidebar: What about security?

New security issues are changes, and all changes require you to do things.

I feel that things that look for local security problems should only alert you if there is something you need to deal with, such as a reliable sign of a breakin.

(Keeping logs of other information is okay, but looking at them is usually just boredom inducing monkey-work; what are you going to do with the information, and is it actually productive?)

Written on 24 June 2005.
« A real use for staticmethod
An open letter to free webmail providers »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jun 24 03:20:47 2005
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.