We've wound up using the spam scores from some other mail systems

August 30, 2017

Like many places, the university has a central email system and all staff and professors have an address there. One of the things you can do with your UTORMail email is forward all of it to an email account elsewhere, such as your account here. A decent number of our users both do this and make a reasonably significant use of their central email address, making it visible and active and thus periodically hit by spammers. When these central email addresses get hit with spam, the central email system forwards it on to us.

Starting a couple of months ago, the university's central email seems to have been targeted by a number of active spam campaigns. People who read their email in the central system were commenting on it and certainly some of our users were reporting it to us, because there was a problem; the commercial anti-spam package we use wasn't recognizing this spam as spam. In theory we could give our normal answer of 'it's a black box, we get what we get', but in practice this felt unsatisfying because the central email system was recognizing and tagging this email as spam before it passed it on to us.

Since the central email system actually uses the same commercial anti-spam package that we do, my assumption was that our lower spam score was because we weren't directly receiving the spam. Since it had gone through the central email system, there was a layer or two of Received: headers and other obscuring things (and the source IPs were different for DNS blocklist checks and so on). It was fairly obvious spam so we were scoring it relatively high, but after the layer of forwarding it wasn't quite sufficiently obvious to get scored as definitely spam.

This was kind of frustrating. The central email system was putting its spam scoring information right there in the message headers of the forwarded messages; we just weren't paying any attention to it. Well, we could change that, so we did (which took a little bit of work in Exim). Now, if we don't score something as spam but the central email system does, we still mark it as spam in the Subject: header, which triggers various downstream processing. In practice, users who are doing spam filtering at all will have this forwarded email go away just the way regular spam does.

(We considered doing something more sophisticated and selective in Exim, but decided that there were simply too many places in our overall mail setup that knew about the Subject: tag, including filtering done by users through procmail and so on. Also, we couldn't think of anything we'd want to do differently depending on who had determined it was spam.)

Since we don't allow a 'not-spam' score from the central email system to override our own opinion, we don't make any attempt to limit this special handling to email that we definitely received from the central email system. This has the useful side effect that if you're forwarding your email through another system before it gets to us, we'll still pay attention to the central email system's spam scoring for you.


Comments on this page:

By Robert Sander at 2017-08-31 05:03:00:

The first email system receiving and identifying the SPAM should just reject it with a 500 SMTP code and never forward it.

Why should a piece of email already identified as SPAM consume resources and even user time on other systems?

And if an email is wrongly identified as SPAM (false positive), the sender gets the bounce from his email system and is able to react. If an email is just "tagged" as spam and maybe even sorted into a "SPAM" or "Junk" folder a false positive will never get read. This may have consequences beyond the technical aspect.

Written on 30 August 2017.
« OpenSSH has an annoyingly misleading sshd error log message
People probably aren't going to tell you when your anti-spam systems are working »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Aug 30 02:24:00 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.