2017-09-04
The idea of 'spam levels' may be a copout
I recently wrote about using the spam scores from another mail system at the university. In a comment, Robert Sander suggested that the original email system should have just rejected the spam at SMTP time. There are a number of issues here, but one of the traditional reasons not to do this is to provide your users with varying levels of spam filtering (which is something we do). This is a perfectly traditional reason, but perhaps this is a copout answer (to be fair, an unexamined one).
The fundamental problem with the idea of spam levels is the usual problem with asking people questions, namely that most people aren't going to be able to make useful decisions because they don't have enough knowledge. With spam scoring levels this is even more of an issue than usual, because many of the answers are either unknowable or take statistical analysis to answer properly. For example, the only reason that I know something about the distribution of the spam scores in our incoming email is because I've gone well out of my way to do analysis on our logs (and I have access to those logs). If I were to ask a user to choose between rejecting on a '70%' spam rating and an '85%' spam rating, how on earth are they supposed to make a sensible choice? At a minimum they'd need to figure out the distribution of spam scores for their email, both legitimate and spam, to see if this is a useful or sensible choice.
In practice there's only one thing that users are going to do with a spam levels knob. They're going to make it more aggressive when they get annoyed with spam and then if they find out that they had false positives (rejecting real email that they want), they might reluctantly make it less aggressive again. Even this represents a failure on our part, since an ideal mail system shouldn't require this tuning in the first place for almost all users.
(The exception are users who get messages that look a lot like spam but aren't. These users will probably always need some way to let through more questionable things.)
So I think there's a serious argument that features like 'spam levels' are essentially a copout. They're our attempt to wash our hands of taking responsibility for the unsolvable problem of rejecting all spam without rejecting anything else. Sure, we can't solve the problem, but we owe it to our users to give it our best shot and then own up to the resulting imperfections as the best tradeoffs we can achieve. Making and justifying this sort of tradeoff is part of what system administration is about in general, after all.
(If we do this, we might also want to think seriously about how we can deal with false positives in an environment where the answer is not 'well, turn your spam filtering off'. Sadly, this probably involves more sophisticated filtering and scoring than is provided by the commercial anti-spam package we use.)
(Some years ago I wrote that filtering spam for users was part of our job, which sort of touches on the same ideas.)