Python won't (and can't) import native modules from zip archives
I've written before about running Python programs from zip files, and in general you can package up Python modules in zip files. Recently I was grumbling on Twitter about the hassles of copying multi-file Python things around, and Jouni Seppänen mentioned using zip files for this but wasn't sure whether they supported native modules (that is, compiled code, as opposed to Python code). It turns out that the answer is straightforward and unambiguous; Python doesn't support importing native modules from zip files. This is covered in the zipimport module:
Any files may be present in the ZIP archive, but only files
.py
and.pyc
are available for import. ZIP import of dynamic modules (.pyd
,.so
) is disallowed. [...]
(The Python 2.7 documentation says the same thing. If you're like
me and had never heard of .pyd
files before, they're basically
Windows DLLs.)
I don't know about the Windows side of things, but on Unix (with
.so
shared objects), this is not an arbitrary restriction Python
has imposed, which is what the term 'disallowed' might lead you to
think. Instead it's more or less inherent in the underlying API
that Python is using. Python loads native modules using the Unix
dlopen()
function, and the dlopen()
manpage is specific
about its API:
void *dlopen(const char *filename, int flags);
Which is to say that dlopen()
takes a (Unix) filename as the
dynamic object to load. In order to call dlopen()
, you must have
an actual file on disk (or at least on something that can be
mmap()
'd). You can't just hand dlopen()
a chunk of memory, for
example a file that you read out of a zip file on the fly, and say
'treat this as a shared object'.
(dlopen()
relies on mmap()
for relatively solid reasons due to
how regular shared libraries are loaded in order to share memory
between processes. Plus, wanting to turn a
block of memory into a loaded shared library is a pretty uncommon
thing; mapping existing files on disk is the common case. There are
probably potential users other than Python, but I doubt there are
very many.)
In theory perhaps Python could extract the .so
file from the zip
file, write it to disk in some defined temporary location, and
dlopen()
that scratch file. There are any number of potential
issues with that (including fun security ones), but what's certain
is that it would be a lot of work. Python declines to do that work
for you and to handle all of the possible special cases and so on;
if you need to deploy a native module in a bundle like this, you'll
have to arrange to extract it yourself. Among other things, that
puts the responsibility for all of the special cases on your
shoulders, not on Python's.
|
|