When Exim generates bounce messages
August 28, 2012
When I started looking at the numbers from our recent spam incident, an inconsistency jumped out at me: we had significantly more bounce messages in the queue than my count of submitted spam messages. Fortunately for my peace of mind, it turns out the cause of this wasn't missing spam messages but that Exim handles bounces differently than I was expecting.
What I was expecting is how I believe that our old mailer behaved; I was expecting Exim to generate a single bounce message at the end of fully processing a message, with all of the addresses that had had errors. In this model of bounces there's at most one genuine bounce generated for each message (plus possibly some number of 'your email has been delayed X amount of time' warning notification messages).
Exim doesn't work that way. Instead, how Exim seems to work is that it generates a bounce message every time it attempts to deliver a message and at least one destination address has an error. If you have a message with a lot of recipients (as many of the spam messages did) where some fail immediately and delivery to others is first delayed and then fails, Exim can generate a whole series of bounce messages from the same original message. Spam runs are probably especially likely to provoke this because they're the situation where remote mail servers are most likely to throttle you back on contact instead of doing anything that would give you an immediate rejection.
(It turns out that Exim logs enough information to actually see this. Some messages in the spam run generated six bounce messages.)
In combination with how Exim does retries, this can have the odd consequence that you can get bounces for addresses that are not on the final delivery list (in as much as you can be said to have a final delivery list in the Exim world). This happens if the bad address was in some expansion (a .forward, a simple mailing list, whatever), the message was not completely delivered to that expansion, and then the expansion was edited to remove the bad address (perhaps because of the bounce message).
I suspect that Exim takes this approach to bounces because it means that Exim doesn't have to save as much information from delivery attempt to delivery attempt. If an address fails it simply has to note that without having to also record why (which may include things like verbose SMTP replies from the other end). All of the verbose information about an address failure can simply be held in memory during a single delivery run, used to generate the bounce, and then implicitly discarded when the delivery process exits.
* * *
Atom feeds are available; see the bottom of most pages.