Wandering Thoughts archives

Importing a Python program that doesn't have a .py extension

In Python, there are a bunch of reasons for having your main program be importable. However, this normally requires that your program have a .py extension and you don't always want to do this. You can always make a copy of the program (or use a symlink) to add the extension when you're working on it, but that can be annoying. Due to writing my entry on why programs should skip having extensions if possible, I wound up wondering if it was possible to do this in Python using things exposed in, for example, the standard library's importlib..

The answer turns out to be yes but you have to go out of your way. The importlib documentation has an example on Importing a source file directly, but it only works for files with a .py extension. The reason for this is covered in the Github gist Importing Python source code from a script without the .py extension; importlib basically hardcodes that source files use a .py extension. However, as covered in the gist, you can work around this without too much work.

Suppose that you have a Python program called "machines", and you want to import the production version to poke around some bits of it. Then:

>>> from importlib.machinery import SourceFileLoader
>>> import importlib.util
>>> import sys
>>> loader = SourceFileLoader("machines", "/opt/local/bin/machines")
>>> spec = importlib.util.spec_from_loader("machines", loader)
>>> machines = importlib.util.module_from_spec(spec)
>>> sys.modules["machines"] = machines
>>> spec.loader.exec_module(machines)

Here I use 'machines' for what the example calls 'module' so that I can avoid an extra 'import machines' to get the program visible under its customary name in my interactive session.

While I'm happy to know this is possible and how to do it in case I ever really need it, this is tedious enough that I'll probably only ever use it infrequently. If I found myself needing to do it somewhat regularly, I'd probably create a personal module (call it, say, 'impfile') that wrapped this up in a function, so I could just do 'import impfile; impfile.get("/opt/local/bin/machines"); import machines'. This personal module could go in either ~/.local/ or in a custom virtual environment that I'd use when I wanted to do this.

(Neither would work if the program itself used a virtual environment, but that's unlikely for the kind of programs I'd want to use this on.)

PS: I think I understand why importlib doesn't have a convenience function to do this or something like it, and I agree with not providing one. In the standard library, it would be too much of an attractive nuisance; it would practically invite people to use it.

(Then they would be sad when a bunch of tools for working with Python code choked on their special files, because the tools are always going to require .py files.)