You want to do spam forwarding on a separate machine

March 4, 2010

I've previously written about how we use a different source IP address when forwarding spam-tagged email than when forwarding other email, so that users who forward all their email can still get some of it. If you do this, it turns out that you really want to use an entire separate machine for this, not just an IP alias on your main mail machine.

(We used a relatively small virtual machine for the purpose, rather than dedicating physical hardware to the job. If you really wanted to, I suppose you could run two instances of the MTA on the same physical machine.)

The problem that pushed us into doing this was queueing and retries. As you might expect, it turns out that spam email gets delayed in very different ways than regular email. Thus, if you send both regular email and spam-tagged email from the same machine you effectively have two mostly separate retry queues, with quite different characteristics, except that your mailer doesn't know this and you probably can't teach it about the real situation. Things get confused from there, in a couple of ways.

First, it makes monitoring and management harder unless you write fairly intricate custom tools; if your mail server has an unusual backlog, is that a real problem or is it just another site that's decided to stop accepting forwarded spam? If it is, can you easily tell which new site has decided to choke on your spam email?

Second, retries for the two different sorts of email get tangled together, both for actual retry times and for writing rules. Most MTAs try to notice when hosts or addresses aren't working and not pointlessly try delivering every piece of email to them only to have it stall. This works when all of the messages for one domain or user run into the same sort of delays, but spam email and regular email can be very different; it's routine for spam to stall when regular email gets through. Since your MTA can't tell the two apart, it's likely to get very confused about what it should do, whether it should back off on delivery attempts, and so on. Equally, you may want to have different retry timings and timeout periods and so on for spam email than for regular mail, but your MTA's retry rules probably can't tell them apart.

(Even when everything was on one machine we were able to make spam email get deleted from our MTA queue faster than the normal timeouts, which is very important for Exim-specific reasons.)

All of these issues went away when we moved spam email to a separate machine and thus a separate queue and mailer configuration. Monitoring became much easier and simpler, the MTA's retry handling was once again sensible, and we could apply different (and slower) retry timings to spam email.

Your separate spam forwarding machine does not need to be very big and fast, since spam forwarding is not a very important job (at least in my opinion) and delivery of such email is not likely to be very rapid in the first place.

In some ways, life would be simpler if you could tell your MTA to put spam email into a separate queue from regular mail (and then keep separate retry information for it, apply separate retry rules, and so on), but I don't know of any current MTA that allows things like that. I may be out of touch with the features of non-Exim mailers here.

Written on 04 March 2010.
« All syndication formats use XML
PCs are (or can be) Unix workstations »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Mar 4 00:15:49 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.