Wandering Thoughts archives


The costs of doing your own system administration automation

There is a constant tension in system administration between doing things by hand in the standard way, versus creating your own custom systems to automate the work. Doing things by hand takes longer and is more tedious, so it sounds like automation should be a slamdunk, but the problem with automation is that you've created a new local piece of software that has to be updated, maintained, learned by other people (especially new people), and understood.

(Or to put it another way, you have taken something that is someone else's problem and made it your own problem.)

Note that the people who originally wrote the systems are going to be bad at assessing how obscure and hard to learn and so on those systems are, because of course the custom systems make perfect sense to them (or at least they're very likely to). To really evaluate this, you need an outsider, such as a new person coming into your group; if they start wanting to strangle you after a while, you have a problem.

This has been a hard lesson for me to learn, partly because I am naturally inclined to be an enthusiastic tool builder and partly because I have spent a great deal of my career as (effectively) the sole sysadmin of an environment. I owe a debt to my co-workers for determinedly prying my eyes open.

(You get a similar 'sole sysadmin' effect if people have very distinct areas of responsibility, where eg one person is the mailer person and no one else really tries to keep up.)

This doesn't mean that you shouldn't ever automate; it just means that you should bear this in mind and make sure to ask yourself if the extra complexity is worth it. Sometimes the answer should be no.

sysadmin/AutomationCosts written at 22:56:24; Add Comment

By day for May 2008: 1 2 3 4 5 6 7 8 9 11 12 13 15 17 18 19 20 21 23 24 25 26 28 29 30 31; before May; after May.

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.