Wandering Thoughts archives

Some DNSBL developments I've just heard about

I mentioned recently that choosing DNS blocklists isn't necessarily a one-time thing that you set and forget. I always knew this in a vague and general way, but I had mostly ignored it until recently. More specifically, until I was writing that entry and wound up looking at the CBL front page, which had a March 24th announcement of news about the PSKY DNS blocklist. To wit, that PSKY had apparently been 'borrowing' Spamhaus data without authorization, that this has been stopped, and that it wasn't clear if they listed anything much any more. We've never deployed PSKY on our main mail server, but I had deployed it on my personal sinkhole spamtrap and it had been having a pretty good hit ratio. 'Had' being the operative word, because starting around the appropriate time I'd not really logged any hits against it.

All of this sent me reading through the rest of the 'Other DNSBLs' portion of the CBL's FAQ. Some of their current opinions match mine (such as Barracuda's public DNSBL being quite aggressive), but others were a surprise to me. Most prominently, the CBL people feel that the current Spamcop BL is now sufficiently safe to use as a general DNS blocklist, where my past experience with it (from several years ago) was that it was too hair-trigger. The rest of the FAQ is interesting in its own way, mostly in that it seems to confirm that there aren't really very many effective DNSBLs any more. Or at least not very many that the CBL feels that they need to talk about.

All we use in our spam filtering is Spamhaus, and I don't think there's much chance that we'll change that. The Spamhaus ZEN is as close as we can get to a high trust, fire and forget DNS blocklist, and even then our users have to opt in to it. But it doesn't hurt to keep an eye on the DNS blocklist landscape every so often (even if there seems to be less landscape than there used to be).

(That diminishing landscape is one reason I'm saddened by the news about PSKY's blocklist. When I first heard of them, they were the first new and effective DNSBL for some time, and frankly we can always do with more good spam-blocking.)

Why modules raising core exceptions mostly hurts, not helps, your users

A while back I wrote an entry about how modules should never raise core Python exceptions. Recently via my Referer logs I found out that some people aren't convinced by my entry, so I feel like taking another run at this topic, this time approaching it from the perspective of someone using your module.

If I'm invoking some function or method from your module and want to trap errors, I need to write code like this:

import yourmod

def fred():
  try:
    res = yourmod.some_thing(10, 20)
  except SOMETHING as e:
    ...

In order to fill in that SOMETHING with the right exception, I need to consult your module's documentation. Given that I have to look this up, reusing a general exception saves me essentially nothing; at most I type a little less, and yourmod.Error versus RuntimeError is not exactly a compelling savings. If I just want to catch your explicitly raised errors, using RuntimeError (or a subclass of it) is not saving me any real effort.

In practice, only catching explicitly raised errors is almost always what people using your module want to do, because of the various dangers of over-broad tries that I mentioned in my original entry and elsewhere. And if I really do want to catch all errors that come out of your code, I can already do that explicitly:

try:
  res = yourmod.some_thing(10, 20)
except Exception as e:
  ...

Notice that raising RuntimeError instead of your own error doesn't actually help me here. If I want to catch all possible errors that can happen during your module's execution, I need to go much broader than merely RuntimeError.

(There are valid cases for doing this broad catching, generally in top-level code that wants to insure that no uncaught exceptions ever surface to the user.)

Which brings me around to the one case where it is sensible to raise standard errors, which is when you're writing code that stands in for standard Python code that raises these errors. This is the one case where using a standard error saves me from looking things up; in fact, using a standard error is generally essential. If you're writing a class that will get used instead of a standard dictionary, raising KeyError and so on is absolutely essential, because that makes your objects transparent substitutes for real dictionaries.

(This is in a sense a matter of API compatibility, where the exceptions that get raised are part of the API. Sometimes this can be explicit, as in the case of KeyError, but sometimes this is more implicit, in cases where raising and catching errors is more uncommon.)

Sidebar: my view on subclassing RuntimeError

I don't think you should do this because I don't think it's useful. If someone is catching RuntimeError today they probably intend to catch significant internal issues inside Python, not errors that your module happens to raise. Sweeping your module's errors into their except clauses is probably not helpful and may even be harmful.

For better or worse, I think that there is (or should be) a strong separation between 'internal' Python errors raised by the CPython interpreter and core Python modules, and module-specific errors raised by modules (yours, third party modules, and non-core modules in the standard library). These two sorts of errors don't really live in the same taxonomy and so putting one under the other is generally not going to help anyone.