Version control comes first

December 21, 2012

A commentator on my last entry wrote (in part):

I also find that sometimes [version control] can fall short in providing context to a change. Good living documentation (wiki, OneNote, etc.) makes up for the area that strict version control cannot; providing the reader with some understanding on context, and "why."

It is my position that most organization can benefit most from a good wiki (or something to that affect), then introduce opportunities for file based version control at a later date.

I very much disagree with this view for both general and pragmatic reasons.

Wikis create (theoretically) living documentation. Documentation is nice, but it is not crucial. (Ask lots of people.)

Version control creates much safer changes (you can see exactly what you changed and you can see how things used to be back when they worked (and you can go back to them)). It is an unusual environment that is not making changes frequently (sysadmins generally exist in large part to make changes) but changes are dangerous. Making changes safer is crucial and a major improvement in your environment.

That is the general reason. Now to the pragmatic one.

At this point in time, any place that's lacking both living documentation and version control is in a bad situation. Either they're so culturally backwards that they haven't been convinced of the virtues of either or they are so badly managed that their sysadmins can't create either. In a place that's backwards, dysfunctional, or both, simple version control is going to be much easier to introduce than living documentation, partly because it's much easier than writing documentation and partly because it can start paying off almost immediately.

Simple version control has a major additional benefit in an organization with problems: it fails much safer. If people stop using simple version control, nothing particularly bad happens (you revert to the status quo ante where you had no change tracking). If people stop updating your living documentation, what you have is misleading, out of date documentation; like incorrect comments in code, this is worse and more dangerous than having no documentation at all.

(This applies in the small as well as in the large, where you don't do a checkin or don't do a documentation update. It's also much easier to catch a missed checkin than a missed documentation update.)


Comments on this page:

From 69.164.167.126 at 2012-12-21 13:20:23:

I suppose this really depends on what we are including in the definition of duties under “Sysadmins” If one’s role is tweaking scripts all day, then cool, spin up a repo in a subversion server or something. Absolutely a must. Trust me, I currently administer a Software Development environment, so I do completely understand how important it really is. But if you are provisioning a brand new system or collection of systems that serve a role, or introducing anything (especially hardware based) that is doesn’t easily fall into a version control mechanism, then there is a gap. IT deals with different ingredients than Software Dev does (hardware, and systems, OSs, apps, etc) that do not lend well to a true version control model. I’ll love the day when the industry is at the point in which everything can fall under revision control, but its not even close to being there yet. I think that we would both agree that at the end of the day, its about understanding what has changed. If that is the essence of the topic of discussion, that I find it difficult to be able to rationalize how “documentation is nice but not crucial” when version control is, especially when we’ve already identified the inherent gaps that come from a lack of being able to apply a version control mechanism to every task that is performed in IT.

By cks at 2012-12-21 17:58:15:

Version control isn't just for scripts and code. It's for any configuration file or control file and in fact any text file that isn't automatically generated and doesn't change so often (by automated or uncontrolled means) that you can't feasibly check it in. We version control code, scripts, the list of our local networks, Apache configurations, DNS zone files, DHCP configurations, VPN passwords (automatically, through a script), and even our master /etc/group.

If you literally have nothing you can version control, well, then I suppose that you can't introduce version control. But this is not the same thing as version control not being the right first step. If you can version control things, you should. As the first step.

From 89.70.184.230 at 2012-12-22 15:28:38:

I agree with Chris. I have invented the very same strategy on my own, to keep config files in a VCS (git, to be specific) -- it was a Postfix server for 300+ users and I didn't have a luxury of having real test environment, changes needed to be applied on a living thing.

As a sysadmin I can tell that there are very little things to be changed with editor that don't fit into VCS. Of course, there are things that are changed using different methods (Zenoss' or Zabbix' configuration, for what I hate them and appreciate Nagios), but those are rather rare in un*x environment.

--
dozzie

From 99.237.23.137 at 2012-12-30 10:34:08:

Prior to working at my last job (which I started in 2003) I'd never used any form of version control on my systems beyond the typical "cp rc.conf bak-rc.conf" type strategy. At the last job, the grandboss was absolutely fanatical about RCSing everything. Now I can't imagine why anybody would want to do it any other way. Your changes are right there, on the system, no web browser or editor or svn/git/hg magic required. True, they're not off-system in case of a catastrophic crash, but that's what backups are for, right?

I agree that living documentation can be nice too, but unless you're the only person working on the system, and you're as fanatical about updating that doc as my former grandboss was about RCS, it will at some point be out of date. If you're not the only person working on the system, the context is gone until somebody notices it's missing and then figures out who was responsible for the change. Hopefully that person remembers why they made the change they did.

-- Mike

Written on 21 December 2012.
« Sysadmins should pretty much version control everything
Packaging in compiled versus interpreted languages »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Dec 21 02:19:11 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.