A gotcha with Python's socket.htonl() and ntohl()

July 2, 2010

Here is something that I ran into the other day: the socket module's htonl() and ntohl() functions will return signed numbers under some circumstances, not unsigned ones, which means that for some input values they will return negative numbers.

(Some quick testing suggests that this happens on 32-bit x86 machines but not 64-bit x86 Linux machines; all of the 32-bit machines I have access to are running some version of Python 2.5. A quick test to see if this is happening to you is checking what socket.htonl(255) returns.)

This is kind of annoying. The underlying C API is specifically documented as returning unsigned numbers and in many circumstances you really need them even in Python, which means that you need to force the results into unsigned yourself.

(This is probably an actual Python bug that thus might get fixed sometime.)

PS: to fix this problem, just mask the results with 0xffffffffL:

M32 = 0xffffffffL
def htonl(n):
  return socket.htonl(n) & M32

def ntohl(n):
  return socket.ntohl(n) & M32

As a side effect, this explicitly forces the results to be Python longs instead of integers. This generally doesn't matter (although I know of one case where it does or did), and besides you don't really have a choice; if you want to represent arbitrary IPv4 addresses as unsigned integers on 32-bit Python machines, you have no choice but to use Python longs.

Comments on this page:

By Dan.Astoorian at 2010-07-02 17:09:27:

(This is probably an actual Python bug that thus might get fixed sometime.)

Or already has been. On an Ubuntu 9.10 32-bit x86 box:

$ python2.5 -c 'import socket; print `socket.htonl(255)`'
$ python2.6 -c 'import socket; print `socket.htonl(255)`'

(The actual Python versions are 2.5.4 and 2.6.4 respectively.)

http://www.python.org/download/releases/2.6/NEWS.txt suggests the change was made in Python 2.6 alpha 1 (released on 29 February 2008).

Note that these functions also no longer accept negative numbers either (you get "OverflowError: can't convert negative number to unsigned long").


Written on 02 July 2010.
« A corollary to the limits of anti-spam precautions
Returning to the era of 'duplicated' Ethernet addresses »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jul 2 01:47:16 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.