What reloading a Python module really does

October 6, 2013

If you're like me, your initial naive idea of what reload() does is that it re-imports the module and then replaces the old module object in sys.modules with the new module object. Except that that can't be right, because that would leave references to the old module object in any other module that had imported the module. So the better but still incorrect vision of reloading is that it re-imports the module as a new module object then overwrites the old module's namespace in place with the new module's namespace (making all references to the module use the new information). But it turns out that this is still wrong, as is hinted in the official documentation for reload().

What seems to really happen is the new module code is simply executed in the old module's namespace. As the new code runs it defines names or at least new values for names (including for functions and classes since def and class are actually executable statements) and those new names (or values) overwrite anything that is already there in the module namespace. After this finishes you have basically overwritten the old module namespace in place with all of the new module's names and binding and so on.

This has two consequences. The first is mentioned in the official documentation: module level names don't get deleted if they're not defined in the new version of the module. This includes module-level functions and classes, not just variables. As a corollary, if you renamed a function or a class between the initial import and your subsequent reload the reloaded module will have both the old and the new versions.

The second is that module reloads are not atomic in the face of some errors. If you reload a module and it has an execution error partway through, what you now have is some mix of the new module code (everything that ran before the error happened) and old module code (everything afterwards). As before this applies to variables, to any initialization code that the module runs, and to class and function definitions.

What I take away from this is that module reloading is not something that I want to ever try to use in live production code, however convenient it might be to have on the fly code updating in a running daemon. There are just too many ways to wind up with an overall program that works now but won't work after a full restart (or that will work differently).

(This behavior is the same in Python 2 and Python 3, although in Python 3 the reload() call is no longer a builtin and is now a module level function in the imp module.)

Written on 06 October 2013.
« The per-NFS-operation data for NFS mounts in /proc/self/mountstats
What happens when CPython deletes a module »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Oct 6 21:59:12 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.