The costs of doing your own system administration automation

May 5, 2008

There is a constant tension in system administration between doing things by hand in the standard way, versus creating your own custom systems to automate the work. Doing things by hand takes longer and is more tedious, so it sounds like automation should be a slamdunk, but the problem with automation is that you've created a new local piece of software that has to be updated, maintained, learned by other people (especially new people), and understood.

(Or to put it another way, you have taken something that is someone else's problem and made it your own problem.)

Note that the people who originally wrote the systems are going to be bad at assessing how obscure and hard to learn and so on those systems are, because of course the custom systems make perfect sense to them (or at least they're very likely to). To really evaluate this, you need an outsider, such as a new person coming into your group; if they start wanting to strangle you after a while, you have a problem.

This has been a hard lesson for me to learn, partly because I am naturally inclined to be an enthusiastic tool builder and partly because I have spent a great deal of my career as (effectively) the sole sysadmin of an environment. I owe a debt to my co-workers for determinedly prying my eyes open.

(You get a similar 'sole sysadmin' effect if people have very distinct areas of responsibility, where eg one person is the mailer person and no one else really tries to keep up.)

This doesn't mean that you shouldn't ever automate; it just means that you should bear this in mind and make sure to ask yourself if the extra complexity is worth it. Sometimes the answer should be no.

Written on 05 May 2008.
« On standard interfaces
The Bourne shell is not a programming language »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon May 5 22:56:24 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.