Automation changes as systems grow

May 2, 2008

Over the time I've worked in system administration, I've observed a common pattern that I will summarize this way:

As your system count grows, you wind up automating the automation.

Here is the levels that I've seen people (myself included) go through:

  • managing systems by hand, one by one.
  • using tools that let you run a command on all of your systems or classes of your systems.
  • automating things for clusters, so that you don't have to explicitly run commands on each system.

There are well-managed places that climb up these levels early, but in many cases it happens because you have no choice; you have enough systems that you can't afford to manage them with any lower level.

(I haven't personally worked in environments that went beyond the third level, but I imagine that when you grow bigger you can't afford to create clusters by hand so you wind up automating the cluster formation and configuration.)

What made this strikingly visible to me was moving backwards in the levels, when I went from an environment where there were enough systems that I was at the third level to an environment where there were few enough systems that they're mostly being managed at the first and second level. It's been somewhat of a culture shock.

Written on 02 May 2008.
« What the co_names attribute on Python code objects is
Why people don't automate sysadmin stuff »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri May 2 23:07:50 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.