Why I want tests to be easy to write

December 21, 2010

One possible answer to my testing problem from the last entry is that I should simply get to work creating the large test data needed. Some times programming involves doing tedious work that we don't like, and this is one of them.

I disagree, because one of my firm opinions is that it should be easy to write (good) tests. What I want is easy automated tests.

Everyone understands why you want the tests to be automated. Having the tests automated means that you will actually run the tests, all the tests, frequently or all the time. Similar logic is why people push for fast tests (the slower your automated tests run, the less often you run them).

As for the easy part, well, I know myself. If it is not easy to write tests, I will write what feels like the minimum number of tests and stop there. The easier that tests are to write, the more likely I am to write tests, in advance, for various exceptional or odd conditions. If writing a test for something odd is a 30 second exercise, I will throw it in just to make sure that my code doesn't have a corner case; I will essentially do exploratory testing. If writing a test is a ten minute slog of tedium, well, I don't go exploring.

This is especially important in something like our ZFS spares system, because a system for deploying spare disks is (pretty much by definition) all about handling exceptional conditions where something has gone wrong. So I actively want it to be easy to check that our spares system handles crazy situations, things that I think are totally implausible but you never know.

The TDD counter argument is probably that what I am doing here is an inefficient attempt to get to 100% code coverage and 100% branch coverage, and what I should really be doing is collecting metrics on that and working towards it. I'm not sure I have an informed opinion on this, but I definitely know that it's more reassuring to be able to write a test for some weird system state that I can think of and actually see the program handling it correctly than it is to just know that the program should handle it right for whatever reason.

(I'm a sysadmin. Trust is not in my nature; verification is.)

Written on 21 December 2010.
« I don't understand how to test complex data structures
The Unix interpreter problem(s) »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Dec 21 01:26:00 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.