Wandering Thoughts archives

2017-08-31

People probably aren't going to tell you when your anti-spam systems are working

As part of yesterday's entry about how we're now using the spam scores generated by the university's central email system, I was going to say that our users seem happy with the results. However, I have to admit that this is not quite true. We don't explicitly know that they're happy; instead, what we know is that they've stopped reporting that they're getting too much spam and can we do something about that.

What I've come to expect is pretty straightforward, namely that users generally aren't going to give us feedback when our anti-spam systems are working well. And why should they? Really, a spam-free email system where all the email they want gets delivered with no false positives is just how things should be, and you don't generally tell people 'good job, the systems are working just how they're supposed to work'. Naturally, people are generally only going to tell you when something goes wrong, either what they think of as an excess of spam or when email they're expecting doesn't get through or gets bounced.

On the one hand, this can be a bit frustrating when (or if) we want to know if some theoretically clever trick we've added to the mail system is making people's email more pleasant. On the other hand, this means that no news is good news; if we're not getting complaints about spam or missing email, we're most likely (still) doing things right. If we change something and nobody says anything, at a minimum our change did no harm.

As a side note, it's probably not reliable to count on users to start complaining if (or when) the amount of spam they see goes up. By now, some number of users have been trained to expect a certain amount of spam in their inboxes, and they won't start complaining out loud until the spam getting through really gets excessive.

(They may click on 'this is spam' or 'mark as spam' buttons if those buttons are conveniently available to them in their mail environment, especially if the buttons appear to do something. If you have such buttons, monitoring how often users click them can likely give you an early warning indicator of increased spam getting through your filters. Or at least email that your users don't want.)

UsersAndAntiSpamFeedback written at 00:52:00; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.