Our current approach for updating things like build instructions

June 8, 2019

At work here, we have a strong culture of documenting everything we do in email, in something that we call 'worklogs'. Worklog messages are sent to everyone in my group, and they are also stored in private, searchable web archive. We also have systematic build instructions for our systems, and unsurprisingly they are also worklog messages. However, they are unusual worklog messages, because they are not only what was done but also what you should do to recreate the system. This means that when we modify a system covered by build instructions, we have to update these build instructions and re-mail them to the worklog system.

For a long time, that was literally how things worked. If and when you modified such a system, part of your job was to go to the worklog archives, search through them, find the most recent build instructions for the system, make a copy, modify the copy, and then mail it back in. If you were making a modification that you weren't sure was final or that we'd want to keep, you had to make an additional note to do this whole update process when the dust settled. If you were in a rush or had other things to do too or weren't certain, it was pretty tempting to postpone all of this work until some convenient later time. Sometimes that didn't happen, or at least didn't happen before a co-worker also modified the machine (with various sorts of confusion possible in the aftermath). A cautious person who wanted to build a copy of a machine for a new Ubuntu version would invariably wind up trawling through our worklog archive to check for additional changes on top of the latest build instructions, and then perhaps have to sort out if we wanted to keep some of them.

At one point, all of this reached a critical mass and we decided that something had to change; build instructions needed to be more reliably up to date. We decided to make a simple change to enable more easy updates; we would commit to keeping the current copy of every set of build instructions in a file in a known spot in our central administrative filesystem, as well as mailing them to the worklog system. That way, we could cut a number of steps off the update process to reduce the friction involved; rather than hunting for the latest version, you could just go edit it, commit it to RCS, and then mail it in to the worklog system.

(The version in the worklog system remains the canonical reference, in part because my co-workers keep printed out copies of the build instructions for various critical systems. An update doesn't really exist until it's been emailed in.)

This modest change is now a couple of years old and I think it's safe to say that it's been a smashing success. Our build instructions are now almost always up to date and it takes much less work to keep them that way. What was a pain in the rear before is now only a couple of minutes of work, often quite simple work. In the common case, you can copy the commands necessary from your existing email message about 'I made the following change to system X', since we always write those when we make changes. As an additional benefit, we don't have to worry about line-wrapping and other mangling happening when we copy email messages around and cut & paste them from the web archive system and so on; the 'real' build instructions live in a text file and never get mangled by any mail-related thing.

In general and in theory I know the power of small changes that reduce friction, but pretty much every time I run into one in practice it surprises me all over again. This is one of the times; I certainly hoped for a change and an improvement, but I didn't have any real idea how large of one it would be.

PS: We also have various 'how-tos' and 'how-this-works' and so on documentation that we keep in the same directory and update in the same way. Basically, any email in our worklog archive that serves as the canonical instructions or explanations for something is a candidate to be enrolled in this system, not just system build instructions.

Written on 08 June 2019.
« Feed readers and their interpretation of the Atom 'title' element
Hardware Security Modules are just boxes running opaque and probably flawed software »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Jun 8 00:18:07 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.