Sending and receiving file descriptors in Python
On some but not all modern Unix systems, file descriptors (the
underlying operating system level thing behind open files, sockets, and
so on) can be passed between cooperating processes using Unix domain
sockets and special options to sendmsg()
and recvmsg()
. There are a
number of uses for this under various circumstances; the one that I'm
interested in is selectively offloading incoming network connections
from one process to another one that is better suited to handle some
particular connections.
In Python 3.3 and later, doing this is simple because
it is directly supported by the socket module. The documentation
even includes code examples for both sendmsg()
and recvmsg()
,
which is handy because they don't exactly have the most Pythonic
of interfaces; instead it's basically a thin cover over the system
call data structures. If you are receiving file descriptors that
are sockets you're still left with the socket .fromfd()
problem.
(I was encouraged to report the socket fd problem as an actual Python bug, where it has quietly been neglected just as I expected.)
Unfortunately Python 2 does not have any support for this in the
socket module, thereby
creating yet another gratuitous Python 2 to Python 3 difference.
Fortunately a number of people have written add on modules to support
this; the ones I found in a casual Internet search are python-passfd, python-fdsend, and sendmsg (which is notably lacking
in documentation). Of these, python-fdsend seems to have the
best API (and is packaged for Debian and Ubuntu); I expect that
it's what I'll use if (or when) I need this feature in my Python 2
code. Note that it doesn't solve the socket .fromfd()
problem.
If you're sending sockets to another process, remember that it is safe
to call .close()
on them afterwards but it is not safe to call
.shutdown()
on them; as I discovered, shutdown()
is a global
operation on a socket and applies to
all file descriptors for it, including ones now held by other processes.
|
|