Towards assessing SORBS' false positive rate
I was somewhat surprised to read in Chris Linfoot's blog that
he uses SORBS,
because I've always considered the top-level
blocklist a little too aggressive. (Considering that I use SPEWS,
this may be a little bit of throwing rocks in glass houses.)
(Update: Chris Linfoot does say that you need a good whitelist to use SORBS.)
Out of curiosity I decided to get a very broad sense of the potential
'false positive' rate for using
dnsbl.sorbs.net as a whole by seeing
how many IP addresses that had successfully delivered email to us
over the past 28+ days were listed in SORBS.
Over this time period, 425 different IP addresses delivered one or
more messages. 27 of them are listed in
dnsbl.sorbs.net; since some
spam mail gets through our blocks, these aren't necessarily all false
positives. Let's take a look at who's included in the roughly 6% of
successful mail deliveries that SORBS would have blocked:
- n10a.bullet.dcn.yahoo.com and several bullet.scd.yahoo.com hosts
- a number of Hotmail machines. Yes, they emit lots of spam, but we do get legitimate email from them.
- two mail.united.com machines
The overall dnsbl.sorbs.net list is a conglomerate of a number of different sub-lists. On checking, all 27 IP addresses were from the 'Spam DB' list, assembled from things that have hit SORBS spamtraps. Most of them are not listed in any other DNS blocklist (some are in blacklist.spambag.org and/or block.blars.org, both of which are very aggressive, a few were in bl.spamcop.net, and one was also in dynamic.dnsbl.rangers.eu.org).
I'm not too surprised by this result, because I consider all automated 'hit a spamtrap and get listed' blocklists to be too dangerous (we don't even do this with our spamtraps locally; for most domains, they only cause email to get deferred).
(While we use bl.spamcop.net, we use it to delay email, not to reject it. The logic behind this is for another entry.)
Needless to say, this is a little too aggressive for us to use here. While we could exempt the important domains we've seen today, there's no certainty that some other important domain we get email from won't briefly have spammer who hits a SORBS spamtrap and then blam. (Given some of the important local ISPs, I'm actually pretty sure that this will happen at some point.)