CSLabRejectionStats-2011-04-26 written at 23:54:48; Add Comment
Mail rejection stats for our external mail gateway
In my recent spam filtering stats, I noted that some spam was rejected before it made it to the spam tagging and filtering system. Well, here's some stats on roughly that; specifically, on how much email our external mail gateway rejects at SMTP time for various reasons. The numbers here are for almost the same seven day time period as the previous stats; there is about a six and a half hour difference in coverage due to when the two systems roll their logs (one does it at midnight, one does it at 6:30am or so).
So, over seven days we:
The two surprises that stand out in this are how frequently spammers
attempt to forge email as from our own domains and how many relay
attempts there are. I'm not terribly surprised that unresolvable
I'm not going to try to estimate the additional 'real' spam volume here,
because in part it depends on your assumptions. For example, should we
consider all email rejected due to unresolvable
(General information on our spam filtering is in CSLabSpamFiltering. While that was written in 2007, almost nothing has changed since then in our setup although I'm sure that the Sophos PureMessage people have been evolving it madly. Such is one of the benefits of outsourcing most of your anti-spam system.)
CslabSpamStats-2011-04-25 written at 00:31:36; Add Comment
A quick look at some spam filtering stats from our system
It's been a while since I thought about generating statistics about what our anti-spam systems are doing and seeing, which probably means that it's about time to do it again. I'm going to look at the past week's statistics, mostly because we upgraded the spam filtering machine recently and we don't have old logs any more. Unfortunately this is not an ideal week to look at, since Friday was a holiday here so the numbers are going to be down from usual.
First, the disclaimers: not all spam makes it to our spam tagging and
filtering system. For example, some people immediately reject email from
IP addresses that are in the Spamhaus Zen list; since this rejects at
So, over the past seven days we saw:
This is well under the level of spam that most sources report. It's possible that our stats are skewed by various things; for example, it may be that most of the active targets of spam have opted in to spam rejection, and so spam to them never makes it to these numbers. (Trying to quantify the volume of rejections is a project for later.)
Our spam system gives messages a spam score from 0 to 100 (with some decimal points of precision allowed; theoretically this is some sort of probability measure). The breakdown of scores is somewhat interesting:
Our current threshold for calling something spam is 60 points or more. These numbers suggest that we could significantly raise the threshold without having a material effect on our spam filtering; on the other hand, since it would have no material effect there seems no reason to do it (other than possibly user perception, and I don't know if users pay any attention to this).
(Note that this is not the same system that I did my old spam stats for, and so if I do regular reports they are going to look different and not be comparable to the old numbers.)
* * *
Atom feeds are available; see the bottom of most pages.