2015-12-13
I still believe in shimming modules for tests
A commentator on my 2008 entry about shimming modules for testing recently asked if I still liked this approach today. My answer is absolutely yes; I continue to love that Python lets me do this, and I feel that it (or a variation on it) is the best approach for mocking out parts of standard modules. There are two reasons for this.
The first is that I continue to believe in what I wrote in my old
entry about why I do this; I would much
rather have clean code and dirty tests than clean tests and dirty
code. I consider code that is full of artificial dependency injection
to be dirty. For instance, it's hard to think of a reason why you'd
need to do DI for socket module functions apart from being able
to inject fakes during testing. Artificially contorting my code to
test it bugs me enough that I basically don't do it for my own
programming.
The second reason follows on from the first reason, and it is that monkey patching modules this way is an excellent way of exactly simulating or exactly replaying the results you would get from them in the real world under various circumstances. If you discover that some tricky real world scenario gives your code problems, you can capture the low level results of interacting with the outside world and then use them for your future tests. You don't need some cooperative outside entity that fails in a specific controlled way, because you can just recreate it internally.
Without some way of doing this 'exact replay' style of injecting results, what at least I wind up with is tests that can have subtle failures. Synthetic high level data can be quietly wrong data, and while synthetic low level data can be wrong too my view is that I'm much more likely to notice because I know exactly what, eg, a DNS lookup should return.
(If I don't know exactly what a low level thing should return, I'm likely to actually test it and record the results. There are ways for this to go wrong, for example if I can't naturally create some malfunction that I want to test against, but I think it's at least somewhat less likely.)
Finally, I simply feel happier if the code I'm testing uses code paths that are as close to what it will use outside of testing. With monkey patching modules for tests, the code paths are authentic right down until they hit my monkey patched modules. With dependency injection, some amount of code is not being tested because it's the code involved with creating and injecting the real dependencies. Probably I will find out right away if this code has some problem, but I can imagine ways to subtly break things and it makes me a bit nervous (somewhat like my issues with complex mocks).