Focusing on what you actually need in a program

June 8, 2010

I recently wrote a local program to do more useful job of reporting the status of our ZFS pools. To simplify slightly, ZFS pools have a number of flavours of redundancy; none, mirroring, RAID-5, and RAID-6 (currently). However, we only use mirrored pools, so for our own uses the program I was writing didn't need to deal with any other sort of pool.

So of course I spent a not insignificant chunk of time making my program handle RAID-5 and RAID-6 pools. And not just handle them, handle them just right, which involved deciding the severity of various faults in them. Maybe someday we will have a use for this and it's certainly correct, but in the mean time it's functionally useless.

This is a perfect illustration of one of my abiding sins in programming (strictly speaking, in development). Left to my own devices, I can easily wind up spending an inordinate amount of time on picky issues that ultimately don't matter for what we need today's program to do. It's hard for me to do otherwise, because I can always see one thing that the program doesn't handle right, or one bit of output that could be improved, or what have you.

(For me this usually manifests as a nearly endless obsession about fine tuning the program's output format.)

Thus, once again, I need to learn to sort out what we need now from the perfect answer to the problem I've set, and to accept that I don't need to handle some cases because they're ones that we don't care very much about or have no use for; it's the core functionality that matters. And when that core works, I need to let go of my feelings that the peripheral things aren't right and need to be polished more. Especially for local tools, the urge for generalism and perfection is a time-wasting mistake (or at least a danger).

It's not an easy thing. Polished, complete code gives programmers a warm feeling of satisfaction (and for good reason in many cases), while there's little joy in code that you know is incomplete and limited. The temptation to spend ten minutes or half an hour or whatever to fix some of those little nagging issues is always there. More than that, I think that such polishing is addictive, partly because it gives you a series of rapid feedback and rewards. It's sort of like unit tests; you can handle a new case or make the output better and ding, you've gotten that feedback. The program is improving before your eyes, never mind that the improvement is unimportant.

(Note that the rules are different for tools that are intended to be general from the start; there it's not a mistake to handle all of the cases, it's part of the requirements. Whether sysadmins should write general tools even when they only need local ones is one of those interesting debates.)

Comments on this page:

From at 2010-06-08 01:55:00:

Might be beneficial to define the problem domain first, maybe very narrowly to begin with. Then do whatever is in scope, and fine tune as long as you want as long as you stay in scope. In theory, you should still feel happy about the outcome as a programmer then.

From at 2010-06-08 09:27:49:

I recently did a similar thing writing Nagios checks. I spent maybe half of the time coding and testing functionality that I knew we didn't want (at least not currently). I included this functionality for two reasons: 1) if we want it later, I figured it would be easier to implement it now than to try to stuff it in later. 2) I hope to get permission to release the checks to the community once it is better tested and someone OutThere(tm) may want it.

In my case (especially having very little programming training), it felt good to do extra work just do make sure it was done right. I do understand your concern though, that doing too much work all of the time is bad. Still, better to be too good than not good enough, right?

By Dan.Astoorian at 2010-06-08 10:31:57:

The more ambitious plan may have more chances of success...provided it is not based on a mere pretension but on some vision of the things beyond those immediately present.

--Inventor's Paradox; George Polya, How To Solve It

Handling cases you don't have an immediate need for is not necessarily a waste. Even if you never need it to handle any other cases, I've written programs that turned out to be more robust for the case it was intended for because I'd made the decision not to pare it down for the specific purpose I wanted to use it for.

For example, if I'm writing a program to parse the output of another program, I find that a generalized version often requires less adaptation later on when a new version of the source program changes its output format slightly.


Written on 08 June 2010.
« One problem with testing system changes
The challenge of analyzing NFS packet traces »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Jun 8 00:17:31 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.