Towards assessing SORBS' false positive rate

January 8, 2006

I was somewhat surprised to read in Chris Linfoot's blog that he uses SORBS, because I've always considered the top-level dnsbl.sorbs.net blocklist a little too aggressive. (Considering that I use SPEWS, this may be a little bit of throwing rocks in glass houses.)

(Update: Chris Linfoot does say that you need a good whitelist to use SORBS.)

Out of curiosity I decided to get a very broad sense of the potential 'false positive' rate for using dnsbl.sorbs.net as a whole by seeing how many IP addresses that had successfully delivered email to us over the past 28+ days were listed in SORBS.

Over this time period, 425 different IP addresses delivered one or more messages. 27 of them are listed in dnsbl.sorbs.net; since some spam mail gets through our blocks, these aren't necessarily all false positives. Let's take a look at who's included in the roughly 6% of successful mail deliveries that SORBS would have blocked:

  • smtp1.newsguy.com
  • mm-retail-out-1102.amazon.com
  • mx3.friendster.com
  • n10a.bullet.dcn.yahoo.com and several bullet.scd.yahoo.com hosts
  • wproxy.gmail.com
  • a number of Hotmail machines. Yes, they emit lots of spam, but we do get legitimate email from them.
  • smtpout0191.sc1.cp.net
  • two mail.united.com machines

The overall dnsbl.sorbs.net list is a conglomerate of a number of different sub-lists. On checking, all 27 IP addresses were from the 'Spam DB' list, assembled from things that have hit SORBS spamtraps. Most of them are not listed in any other DNS blocklist (some are in blacklist.spambag.org and/or block.blars.org, both of which are very aggressive, a few were in bl.spamcop.net, and one was also in dynamic.dnsbl.rangers.eu.org).

I'm not too surprised by this result, because I consider all automated 'hit a spamtrap and get listed' blocklists to be too dangerous (we don't even do this with our spamtraps locally; for most domains, they only cause email to get deferred).

(While we use bl.spamcop.net, we use it to delay email, not to reject it. The logic behind this is for another entry.)

Needless to say, this is a little too aggressive for us to use here. While we could exempt the important domains we've seen today, there's no certainty that some other important domain we get email from won't briefly have spammer who hits a SORBS spamtrap and then blam. (Given some of the important local ISPs, I'm actually pretty sure that this will happen at some point.)

Written on 08 January 2006.
« Some notes on Solaris 9's Sunscreen IP filtering package
Weekly spam summary on January 7th, 2006 »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jan 8 01:43:19 2006
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.