The consequences of importing a module twice

May 11, 2013

Back when I wrote about Python's relative import problem, I mentioned that only actually importing a module once can be important due to Python's semantics. Today I feel like discussing what these are and how much they can matter.

The straightforward thing that goes wrong if you manage to import a module twice (under two different names) is that any code in the module gets run twice, not once. Modules that run active code on import assume that this code is only going to be run once; running it again may result in various sorts of malfunctions.

At one level, modules that run code on import are relatively rare because people understand it's bad form for a simple import to have big side effects. At another level, various frameworks like Django effectively run code on module import in order to handle things like setting up models and view forms and so on; it's just that this code isn't directly visible in your module because it's hiding in framework metaclasses. But this issue is a signpost to the really big thing: function and class definitions are executable statements that are run at import time. The net effect is that when you import a module a second time the new import has a completely distinct set of functions, classes, exceptions, sentinel objects, and so on. They look identical to the versions from the first import but as far as Python is concerned they are completely distinct; fred.MyCls is not the same thing as mymod.fred.MyCls.

(This is the same effect that you get when you use reload() on a module.)

However, my guess is that this generally won't matter. Most Python code uses duck typing and the two distinct classes are identical as far as that goes. Use of things like specific exceptions, sentinel values, and imported classes is probably going to be confined to the modules that directly imported the dual-imported module and thus mostly hidden from the outside world (for example, it's usually considered bad manners to leak exceptions from a module that you imported into the outside world). In many cases even the objects from the imported module are going to be significantly confined to the importing module.

(One potentially bad thing is that if the module has an internal cache of some sort, you will get two copies of the cache and thus perhaps twice the memory use.)


Comments on this page:

From 83.157.48.229 at 2013-05-12 02:55:44:

I am very surprised by your post because it is not what I observe:

Just consider this module (put it in testimport.py)

def create_const():
    import random
    return random.random()
class Test(object):
    my_static = create_const()
    def __init__(self):
        self.my_local = create_const()
        print "static= %f" % self.my_static
        print "local = %f" % self.my_local

then use it via 2 imports:

import testimport as ti1
t1 = ti1.Test()

and

import testimport as ti2
t2 = ti2.Test()

you have back

>>> static= 0.982529
>>> local = 0.620927

and

>>> static= 0.982529
>>> local = 0.435612

the first variable being the same, meaning that t1 and t2 are pointing o the same module that is once in memory only, no?

From 88.96.1.126 at 2013-05-12 16:47:30:

Anonymous commenter, you are confusing the local name used by the importer (t1 or t2 in your example) with the global name used in sys.modules (testimport in your example).

Try this:

$ mkdir -p foo/bar
$ touch foo/__init__.py foo/bar/__init__.py
$ mv testimport.py foo/bar/

>>> import sys
>>> sys.path.append('foo')
>>> import foo.bar.testimport as ti1
>>> import bar.testimport as ti2

- Ben Hutchings

Written on 11 May 2013.
« Illustrating the tradeoff of security versus usability
The Unix philosophy is not an end to itself »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat May 11 22:16:08 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.