Wandering Thoughts archives

Replacing modules for testing (and fun)

A while back I wrote about how I monkey-patch Python modules for testing, selectively replacing various functions they provide with test ones that return the results I like. But it's not the only way to do this. If monkey-patching system modules this way makes you nervous (perhaps due to its potential side effects), there's another way to do it: you can entirely replace the system module itself in the module you're testing.

After all, imported modules are (more or less) just entries in the module's namespace, and you can reach in to another module's namespace and replace things there; that's how you monkey-patch things in general in Python. So if you are testing your dog module and it uses socket, you can change where dog.socket points pretty much just as easily as you can change where socket.gethostbyaddr does.

(Note that neither sort of replacement deals very well with 'from <module> import ...', which will force you to hunt down and re-point multiple names in the test module's namespace.)

The replacement doesn't have to be a module, either; all you need is something that has a namespace. The easiest thing to use is an instance of a class, which also conveniently lets you put all of your fake module functions and so on in one spot (and store state information). If you want you can even give your class a __getattr__ method and forward all things that you don't implement to the real module.

(If you want to be sure that the module under test only calls things that you know about, you can restrict what gets forwarded.)

There are advantages and drawbacks to each approach. Shimming module functions takes more work and has global effects, which may be good or bad depending on what you're testing. Replacing modules has strictly local effects and is probably simpler if you need to replace a bunch of things or keep state between your various replacement functions.

Sidebar: an example fake socket module

import socket

class FakeSocketModule(object):
    def gethostbyaddr(self, ip):
        return ('www.google.com', [], [ip,])
    def __getattr__(self, name):
        return getattr(socket, name)

This fakes out socket.gethostbyaddr() (in a very simple way) and forwards everything else to the real socket module.