How Exim makes traditional .forward semantics work

July 7, 2011

Traditional .forward semantics allow you to put your own address in your .forward; this means 'deliver to me, bypassing my .forward'. As a mailer construction kit, Exim doesn't have any specific support for handling .forwards; it has some generic features that you can build .forward handling out of. As a consequence of this, it doesn't have any specific handling for this odd bit of .forward semantics and instead supports it in a generic way. I've mentioned this before in an entry on the power of Exim routers but I just pointed to the official Exim documentation for details, and the official documentation is a little bit opaque.

Each message that Exim handles starts out with some number of top level addresses, each of which is routed separately. In the process of doing this, individual routers may replace the current address with one or more new addresses (through, for example, expanding a .forward). Exim then normally tries to recursively route these new addresses just as if they were top level addresses, although it keeps track of the fact that they are 'children' of some address.

(With aliases and simple mailing lists and .forwards that forward mail to people who also have .forwards, you can have a many level chain of descendant addresses that were created from a single top level address.)

When Exim is doing this recursive routing for a particular top level address, it remembers which routers have already handled which addresses. Then if the address currently being routed is the same as one of its ancestor addresses and the ancestor address has already been processed by a particular router, Exim skips that router, acting as if the router was inapplicable to the address or wasn't there at all (instead of having the router re-process an address that it has already processed once); processing the address will fall through to the next router (or routers). In a typical Exim configuration, what's next after the router that handles .forwards is the router that sends people's mail to /var/mail/<user>.

This skipping of routers has to happen separately for each top level destination. If an email message is sent both directly to cks and to sysadmins, an alias that cks is on, you don't want cks to have one copy of the message handled by his .forward and another copy wind up in /var/mail/cks. Also, this skipping of routers is is completely separate from how Exim merges several copies of the same destination together and does only a single delivery to each unique destination (so that in this case cks's .forward will handle only one copy of the message).

(In fact the check has to be separate for each chain of address expansion. We need to be sure that this skipping is only triggered for genuinely recursive addresses and routers.)

In theory this skipping of routers applies to any type of router. In practice only a few of Exim's various types of routers can replace addresses with new addresses and so can possibly trigger this; most of the routers simply give destinations for addresses. At the same time, nothing restricts this to only happening to your router for .forwards; for example, an accidental alias loop will cause the alias handling router to be skipped in a similar way and the results there could be a lot more odd and peculiar (I suspect that one common result would be a 'no such user' error in addition to the message getting delivered to everyone on the alias).

One corollary of all of this is that it's potentially dangerous to create an address-expanding router that returns different results depending on stuff that can change during address routing; for example, a router that returns a different expansion based on the envelope sender address. Such a router won't get invoked a second time on the same address in a recursive situation, even if it would have returned a different, non-looping result. In its loop-breaking behavior, Exim implicitly assumes that every router return the same thing when recursively invoked on the same address.

(Exim does not literally memoize the result of evaluating the router for a given address, although it does cache and memoize the result of a lot of lookups that routers do.)

Sidebar: one way to get an alias loop

Suppose that you have a generic group alias, and a member of the group is going to be away. They think 'I know, I'll forward my email to the generic group alias to make sure that things get handled even if people email me directly'. The pernicious thing is that this appears to work if they test by mailing themselves, because then it's a .forward loop; the incoming mail goes .forward → alias → .forward, the .forward is skipped the second time around, and it all looks good. Only when the group alias is emailed directly does it become an alias loop (going alias → .forward → alias). Pick a rarely used group alias and it could be a while before this blows up.

PS: if you want to catch this in an Exim configuration, I think what you want is a second router that applies to all aliases and just errors them out with 'alias loop detected'. Assuming that both routers accept the exact same set of addresses in the same situations, the only time this second alias-handling router can trigger is if the first one is skipped for some reason, and generally the only way that that can happen is in the situation above, ie there's a loop.

(Disclaimer: I just came up with this idea and haven't actually tested it.)

Written on 07 July 2011.
« Our solution to the spam forwarding problem
An interesting gotcha with Exim and .forward processing »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jul 7 00:49:29 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.