Chris's Wiki :: blog/programming/SocketListenLimitation

An irritating limitation of listening sockets

March 1, 2007

Traditional sockets implementations have an irritating limitation, at least for TCP sockets: it is impossible to gracefully and cleanly shut down listening (server) sockets. The problem is that kernels accept new TCP connections for listening sockets without asking your server; accept() doesn't actually accept the connection, it just gives you the next one that the kernel has already accepted on your behalf.

This means that to shut down cleanly, you need a two-stage process: first you tell the kernel to stop accepting new connections for you, and then you process all of the existing accepted connections. If you simply dump everything, clients will see abrupt disconnections of established connections, because from their perspective the connection is complete; their connect() finished, and they can even have been sending data.

Unfortunately, there's no two-stage close interface for server sockets (at least, none that I can see):

if you just close() the socket, you flush the pending accept() queue.
you can't un-listen() by setting the backlog size to 0 or -1 or the like.
shutdown() is either the same as close(), does nothing, or produces very peculiar results, depending on the system and the exact arguments; in no case does it cause further connect() calls from clients to be refused.
SO_ACCEPTCONN is a read-only socket option.

(You can do this with Unix domain sockets, by removing the socket file itself without closing your server socket fd.)

Among other consequences, this means that protocols where the client connections and immediately starts sending data are dangerous; the client had better be prepared for the entire conversation to fail. The only way to make sure that the server is really there (instead of in the process of shutting down) is to wait for it to send you something, which may be why so many Internet protocols start with greeting banners from the server.

(As a pragmatic matter the server can lower the risk by not closing things down until accept() on a non-blocking socket returns an error, but there's still a concurrency race, just a smaller one.)

Comments on this page:

By Dan.Astoorian at 2007-03-02 12:22:46:

Even if you could tell the socket to stop listening for new connections, there may be connections for which the server has already sent SYN-ACK but for which the client has not yet completed the handshake. From the client's perspective, the connection has been fully established, but the server cannot yet accept() it.

This means that to properly implement the semantics you're looking for, the application would need to be able to wait for half-open connections to either complete or time out before it could conclude that it had handled all TCP connections that the kernel had accepted. (E.g., accept() would probably have to report something like EAGAIN while half-open connections existed and something like EINVAL thereafter, and select()/poll() on the listening socket would have to signal an error after the last half-open connection either timed out or was accepted.)

It is indeed much simpler to have the protocol running on top of TCP indicate acceptance of the connection at the application level; this is a facility that most application protocols should have anyway, since there can be other reasons to abort the connection after they've been accepted at the network level (e.g., failed tcpwrapper checks, or failures while trying to launch the process or thread to handle the connection).

--Dan

Written on 01 March 2007.

«	Using Unix domain sockets from Python

A story of network weirdness