Wandering Thoughts archives

Python's relative import problem

Back in this entry I bemoaned the fact that Python's syntax for relative imports ('from . import fred') is only valid inside modules. The reason to have it valid outside modules is fairly straightforward; it would allow you to import and run the same Python code whether or not you were doing 'import module.thing' from outside the module's directory or sitting inside the module's directory doing 'import thing'. The way things are in Python today, once you start using relative imports in your code it can only be used as a module (which has implications for it being somehow on your Python path and so on even while you're coding).

Unfortunately for me, I suspect that this restriction is not arbitrary. The problem that Python is probably worrying about is importing the same submodule twice under different names. The official Python semantics are that there is only one copy of a particular (sub)module and its module level code is run only once, even if the module is imported multiple times; imports after the first one simply return a cached reference.

(These semantics are important in a number of situations that may not be obvious, due to Python's execution model.)

However, Python has opted to do this based on the apparent (full) module name, not based on (say) remembering the file that a particular module was loaded from and not reloading the file. When you do a relative import inside a module, Python knows the full name of the new submodule you're importing (because it knows the full, module-included name of the code doing the relative import). When you do a relative import outside a module, Python has no such knowledge but it knows that in theory this code is part of a module. This opens up the possibility of double-importing a submodule (once under its full name and once under whatever magic name you make up for a non-module relative import). Python opts to be safe and block this by refusing to do a relative import unless it can (reliably) work out the absolute name.

(There are still plenty of ways to import a module twice but they all require you to actively do something bad, like add both a directory and one of its subdirectories to your Python path. Sadly this is quite easy because Python will automatically add things to the Python path for you under some common circumstances.)