An interesting gotcha with Exim and .forward processing

July 7, 2011

Yesterday I described how Exim implements traditional .forward semantics where putting your own address in your .forward means 'deliver it to me, bypassing my .forward'. Because Exim is a mailer construction kit, this isn't a specific feature for .forward handling, it's a generic general feature that happens to give you this result.

So far, so good. Now, let's talk about our .forward-nonspam feature. In the abstract, this is just another .forward-style router that reads a different file and only triggers under some conditions. In concrete, we need several routers in sequence, each of them doing one step of the processing logic:

  1. if .forward-nonspam exists and the message is not spam, expand .forward-nonspam
  2. if the message is spam, .forward-nonspam exists, and .forward does not exist, discard the message
  3. if .forward exists, expand .forward

If you have both a .forward-nonspam and a .forward, the third rule will only be triggered for spam messages because your .forward-nonspam skims off non-spam messages first.

Well. Mostly. You see, although all three of these routers are conceptually a single block of .forward processing, Exim doesn't know this; as far as Exim is concerned, they are three separate and completely unrelated routers. Now suppose you put your own address into .forward-nonspam and also have a .forward, as you might do to create a simple 'put all non-spam email into my regular inbox and all spam mail into a file' system, and you get a non-spam message. Exim processes things until it reaches the first router, expands your .forward-nonspam, gets your address and restarts routing it, gets to the first router again, sees that the router has already handled this address, and only skips that router, not all three .forward-processing routers. So your address falls through to the third router, which says 'sure, you have a .forward, I'll handle this' and dumps the non-spam message into the file for spam email.

Oops.

The fix for this is to split the third router into two routers, one for the case where you do have a .forward-nonspam (where it would only handle messages that are explicitly spam-tagged) and a second one for the case where you have no .forward-nonspam (where it would handle everything). However, this requires an annoying level of repetition in the Exim configuration file.

(For technical reasons I think that you can't combine this together in a single condition on a single router that works quite exactly right.)

Sidebar: the technical reasons

The condition you need is 'if .forward exists and either .forward-nonspam doesn't exist or the message is non-spam'. Exim has special support for securely and correctly checking for file existence over NFS, but this support is only available in the require_files router condition. However, we need to use a condition check with a '${if ...}' string expansion to check 'is non-spam'. You can't or together separate router conditions (they are all implicitly and'd together instead), and the does-file-exist check that's available in a ${if expansion doesn't work the right way over NFS.

In theory you could get around this with various evil hacks involving Exim string expansion, maybe.

(Talking to myself: one could rephrase the condition as 'if .forward exists and, if the message is non-spam, .forward-nonspam doesn't exist' and then write this as a single require_files condition with a conditional string expansion in it.)


Comments on this page:

From 97.107.130.220 at 2011-07-08 16:25:09:

I think this is doable with just your 3 rules simply by swapping them?

  1. discard if spam and .forward-nonspam and not .forward
  2. expand .forward if .forward
  3. expand .forward-nonspam if ham and .forward-nonspam

This will redundantly expand .forward for ham before deciding to expand .forward-nonspam instead. But all mail ends up in the right mailbox – no?

Aristotle Pagaltzis

By cks at 2011-07-08 20:54:25:

That wouldn't work. Consider the following setup:

; cat .forward-nonspam
cks
; cat .forward
/u/cks/Mail/spam

If we process .forward for ham as well as spam, /u/cks/Mail/spam winds up with a copy of all messages instead of just the spam messages.

(In Exim, processing .forward this way would also consume the address so it would never reach either .forward-nonspam or the further router that delivers the message to your inbox.)

From 97.107.130.220 at 2011-07-09 09:43:08:

So the first matching rule short-circuits further processing unless it also yields an identity result, in which case it restarts the processing but with itself exempted?

Aristotle Pagaltzis

By cks at 2011-07-09 20:47:12:

Unless you set a special router option, an address is only ever processed by one router, the first one whose conditions all apply. The type of router used for .forwards (and for aliases, and for simple mailing lists, and so on) can generate new addresses; when this happens, the new addresses are routed from scratch as completely separate addresses. On this re-routing pass, Exim does the whole complex 'skip a router in certain situations' thing.

(There can be multiple levels of generating new addresses in this whole routing and re-routing process; an address might be an alias, which expands into a simple mailing list, which includes a second mailing list, which includes someone who has a .forward, and so on.)

Written on 07 July 2011.
« How Exim makes traditional .forward semantics work
My view on iSCSI performance troubleshooting »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jul 7 15:49:07 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.