Keeping changing systems stable
Back in June, I wrote about how unchanging systems should be stable. There is an important consequence of this:
To control stability, you must control change.
(This clearly follows from change being the only thing can destabilize your unchanging stable systems.)
I think that this is subtly different from taking changes carefully because the changes themselves could possibly be broken. This is the realization that any change may perturb the overall system, requiring it to be restabilized; it is the need to control what one has defined as the root cause of instability.
(There are certainly ways to structure systems so they are resilient in the face of change, although I'm not sure if this is a well understood area.)
This also gives me a new perspective on a lot of sysadmin twitches about change, especially change that's not under our control; for example, automatically applied vendor updates. It's not just that vendors might release updates with problems, it's that having a system change unpredictably all the time makes stabilizing it that much more difficult.