Wandering Thoughts archives

An advantage of virtualization for license servers

One of the more subtle advantages of full hardware virtualization is that your virtualized hardware always stays the same, no matter what actual hardware you happen to be running on. This is a real advantage for license servers, which are often nodelocked with various crazy schemes that assume you are trying to cheat on them if you change hardware.

(Sure, the company might fix the problem for us. If they're still in business, still interested in our business, still having anything to do with that ancient version of the software, and so on. All too often, companies seem to treat license problems as a great opportunity to extract more money from you.)

Virtualized hardware can even survive nodelocking schemes that require the inode numbers of crucial files to not change, because the disk is virtualized too. All you have to do is back up and restore the disk image (which is generally just some files), and everything comes back just as it was, inode numbers and all.

In fact, simplifying backup and restore in general is a nice feature of virtualized hardware. We no longer have to wrestle with various system specific programs that may or may not completely work; instead we can just snapshot, back up, and restore some files, something for which we have lots of tools. And we're guaranteed to back up and restore everything, with complete fidelity. And the whole process is much simpler, especially restoring a system (which can often be a nightmare).

The other great advantage of static hardware is static operating system stuff that depends on hardware, which in turn means that we never again have to do things like dig up a nearly exact replacement for a piece of dead hardware in order to get a system running again in a hurry without monkeying madly with its configuration. This is especially an issue for old systems, where you may not be able to reinstall the operating system on your recent hardware even if you want to because it lacks the necessary drivers.

Please leave IOError and OSError alone

From Lawrence Oluyede's blog entry about recent Python SVN updates:

• urllib raises IOError if the servers response contains an invalid HTTP status line.

Augh. Wait, that's not strong enough, let me try again: AUGH.

In fact, it gets worse. To quote from the urrlib2 SVN source code's comment about its URLError error class:

URLError is a sub-type of IOError, but it doesn't share any of the implementation. [...]

(urllib2 also directly raises mis-formatted IOError exceptions for some things.)

Python's IOError and OSError have a very specific purpose; they report low level operating system errors. That is the only thing they should be used for, most especially if you are not going to carefully mimic the existing exception object format (which you can't do, because you don't have a real errno value). Having an alleged IOError object that looks nothing like real IOError objects is actively offensive, especially when it is in the standard library.

What these people have done, whether they realize it or not, is to turn IOError exception objects into nothing more than strings, because these modules have insured that there is no higher structure that you can count on an IOError instance having. Unfortunately the relevant Python documentation does not mention this, and continues to mislead the innocent into believing that when they catch an IOError exception they can do useful things like look at its errno or strerror attribute.

Or, to boil it down: never raise or subclass IOError, OSError, or EnvironmentError.

(This applies to EnvironmentError as a whole because using it in an except clause is the common way to catch both IOError and OSError at once.)

The only exception is if you are somehow directly calling a C library function (that is documented as setting errno on failures, which not everything does), in which case you should do the research to exactly mimic how the Python core code does it.

To my horror, urllib2 is hardly the only offender; there are even some in C level modules (in bz2module.c and zipimport.c). But that doesn't excuse them making the situation worse; in fact they should be moving towards making the situation better. Even with backwards compatibility concerns they could have raised URLError instead of IOError directly (or taken the opportunity to introduce a proper error class if they felt URLError was inapplicable).

(This rant has been brought to you by my attempt to catch up on Planet Python.)

Weekly spam summary on March 31st, 2007

This week, we:

• got 12,348 messages from 259 different IP addresses.
• handled 17,799 sessions from 1,226 different IP addresses.
• received 171,239 connections from at least 53,794 different IP addresses.
• hit a highwater of 28 connections being checked at once.

Somewhat to my surprise, volume is down again from last week, although the concurrent connections highwater is up a lot.

Kernel level packet filtering top ten:

Host/Mask           Packets   Bytes
68.230.241.0/24       21236   1031K cox.net
213.29.7.0/24         17748   1065K centrum.cz
205.152.59.0/24       15832    718K bellsouth.net
68.230.240.0/24       13549    658K cox.net
213.4.149.12          13506    702K
68.168.78.0/24         9359    449K adelphia.net
72.249.13.82           8992    494K
72.249.13.84           5195    286K
72.54.120.137          4553    219K
72.249.13.83           4264    235K

By contrast, our kernel packet filtering blocks are up significantly from last week, partly because I was aggressive about throwing blocked advance fee fraud webmail sources into the kernel filters early on. As a result of this, blocked webmail sources account for half the top ten, and all four of the top spots. (To save space, I've just annotated the main listing with who each /24 belongs to.)

• 213.4.149.12 and 72.249.13.82 return from last week.
• 72.249.13.84 and 72.249.13.83 kept trying to send us email from the user name do_not_reply at a domain with temporary DNS failures. For that extra encouragement to accept their email, the machines HELO'd as otcpicksnews4.com and otcpicksnews3.com respectively.
• 72.54.120.137 kept trying with a bad HELO name.

A note to people: if you want to look straightforward and innocent, don't give your machines separate domain names that vary only in their trailing digits, and especially don't try to send email from them with the same SMTP MAIL FROM. Because there are really not that many innocent explanations for why you would need your outgoing email pool machines to have different domain names.

Connection time rejection stats:

  48510 total
  26705 dynamic IP
  16564 bad or no reverse DNS
   3814 class bl-cbl
    204 class bl-sbl
    180 class bl-dsbl
    161 acceleratebiz.com
    110 class bl-pbl
    109 dartmail.net
     74 cuttingedgemedia.com
     71 class bl-njabl
     43 class bl-sdul

The highest SBL sources this week are a rerun of last week: first SBL52715 with 149 rejections, then SBL50181 with 22. I'd find it striking, but mostly I find it depressing.

Seventeen of the top 30 most rejected IP addresses this week were rejected 100 times or more; the leader is 201.244.113.194 (237 rejections, for having bad reverse DNS). Eleven of the top 30 are currently in the CBL, 3 are currently in bl.spamcop.net, eight are in the PBL, and a grand total of 16 are in zen.spamhaus.org.

(Locally, 18 were rejected as 'dynamic IP', 11 were rejected for bad or missing reverse DNS, and one was a cuttingedgemedia.com machine.)

This week's Hotmail numbers are:

• 3 messages accepted, at least two of which were almost certainly spam.
• no messages rejected because they came from non-Hotmail email addresses.
• 36 messages sent to our spamtraps.
• 1 message refused because its sender address had already hit our spamtraps.
• 2 messages refused due to their origin IP address (one in SBL44668, a listing from August 12th 2006, and one from Burkina Faso).

And the final numbers:

Well, so much for the nice numbers on bad bounces from last week. In better news, no particularly source of bad HELOs stood out, and the most active one, 74.62.160.117, only had 52 rejections.

Bad bounces were sent to 22 different bad usernames this week, with the most popular being noreply with 10 attempts. Most of the bad usernames were at least not random, and some of them were for past local users; there were a fair number of usernames like trinawebber that were trying for plausible first name plus last name.

The most prolific source of bad bounces is an ISP in Bulgaria, followed by Earthlink. The remaining bad bounces come from all over, including a well-named machine called 'mail.victim.com' (72.1.148.50).