Wandering Thoughts: Recent Entries For 2010/07/02

A gotcha with Python's socket.htonl() and ntohl()

Here is something that I ran into the other day: the socket module's htonl() and ntohl() functions will return signed numbers under some circumstances, not unsigned ones, which means that for some input values they will return negative numbers.

(Some quick testing suggests that this happens on 32-bit x86 machines but not 64-bit x86 Linux machines; all of the 32-bit machines I have access to are running some version of Python 2.5. A quick test to see if this is happening to you is checking what socket.htonl(255) returns.)

This is kind of annoying. The underlying C API is specifically documented as returning unsigned numbers and in many circumstances you really need them even in Python, which means that you need to force the results into unsigned yourself.

(This is probably an actual Python bug that thus might get fixed sometime.)

PS: to fix this problem, just mask the results with 0xffffffffL:

M32 = 0xffffffffL
def htonl(n):
  return socket.htonl(n) & M32

def ntohl(n):
  return socket.ntohl(n) & M32

As a side effect, this explicitly forces the results to be Python longs instead of integers. This generally doesn't matter (although I know of one case where it does or did), and besides you don't really have a choice; if you want to represent arbitrary IPv4 addresses as unsigned integers on 32-bit Python machines, you have no choice but to use Python longs.

A corollary to the limits of anti-spam precautions

One of the corollaries to the limits of certain sorts of anti-spam heuristics is that if you see spam software actively working to get around any particular sort of precaution, you can assume that the particular precaution is getting reasonably popular.

This follows from spammers being lazy but not stupid; if the precaution was not reasonably popular, dealing with it probably wouldn't be worth the time of the programmers behind the spam bots. That they do deal with it implies that it is worth their while to do so, which suggests that it is reasonably popular. The harder the precaution is to deal with, the more this holds.

(My strong assumption these days is that essentially all spam software is being written for cold-blooded commercial reasons and so its programmers write the features that will make them the most extra money.)

Of course, all bets are off if you annoy the programmer of some spam software. But this probably isn't very likely for most people.