Why mail systems should not defer rejections to RCPT TO time

October 20, 2007

There is a movement for the default configurations of things like exim to defer sender verification to RCPT TO time; instead of reporting an error or a defer after the MAIL FROM, all MAIL FROMs are accepted and only later does the message start getting errors. I have recently come to a realization about why this is wrong, and I even have an example.

The problem is that when you give at least a 4xx error to an RCPT TO, it makes the sending mailer think that there is a problem with that RCPT TO address, not with the MAIL FROM address. The sending mailer may then sensibly defer all email to that recipient, because after all you told it that there was a problem with that address. (The actual text of your 4xx error may explain the situation, but mailers don't yet read English error messages.)

We have actually seen this happen with email from our central campus mail system for someone who was forwarding their email to our system. Some spam domain fell out of the DNS between the central mail system accepting it and it coming to us, we started giving temporary defers at RCPT TO time, and all mail for this person backed up.

I believe that this is done because people feel that some mailers do not react well to MAIL FROM errors (and I've occasionally seen evidence of that in our logs). However I feel that the cure is worse than the disease, and such bad mailers are clearly violating the specification to start with; coddling spec-violating mailers while causing problems for mailers that are following the spec does not seem like a good tradeoff for me.

(Besides, we ran our system with sender verification problems reported at MAIL FROM time for years without getting any complaints or problem reports, so we have empirical evidence that it works fine.)

Theoretically this also allows you to accept mail for postmaster whether or not the sender address actually exists. Personally I do not believe that this is actually a feature, especially since it has been years since we got any legitimate outside email to postmaster; what we do get has been spam.


Comments on this page:

By Dan.Astoorian at 2007-10-22 14:55:01:

when you give at least a 4xx error to an RCPT TO, it makes the sending mailer think that there is a problem with that RCPT TO address, not with the MAIL FROM address. The sending mailer may then sensibly defer all email to that recipient, because after all you told it that there was a problem with that address.

I see nothing in the specs that justify such an assumption. (In fact, RFC 2821 itself even provides at least one counterexample: it recommends a server response code of 452 if the server's site-policy limitation on the number of RCPT commands in a single SMTP transaction is exceeded.)

Theoretically this also allows you to accept mail for postmaster whether or not the sender address actually exists.

...or whether or not the sender address or connecting host is blacklisted. There is considerable merit in allowing blacklisted sites to contact you to let you know that you've blacklisted them in error.

Of course, if you're rejecting the message due to blacklisting, including the sender address definitively not existing, you're better off doing it with a 5xx response code; 4xx should only (if ever) be returned if there actually was a temporary failure trying to determine whether the address was acceptable and a retry might actually succeed.

(It's probably reasonable to assume that blacklisted addresses will stay blacklisted in the short term; it's too inefficient to classify a failure as temporary on the off-chance that the sender address might get whitelisted by the next delivery attempt.)

This begs the question, for such mailers that defer all messages in response to a 4xx response to RCPT TO: if it got a 5xx response code instead, would it bounce all messages for that address?

--Dan

By cks at 2007-10-27 23:25:14:

My feelings on blacklist appeals are verbose enough to make an entry, BlacklistAppeals.

All our definite rejects are 5xx errors (including blacklisting). The specific case where this issue came up was a 'temporarily' unresolvable MAIL FROM domain, which has to get a 4xx error.

I believe that the RFCs are silent on MTA retry policies (except to say that 5xx errors should not be retried and 4xx errors should be), which is sensible; retry policies are really beyond their scope.

I don't know if this particular mailer would auto-bounce further email to an address on a 5xx failure without trying a SMTP session; it's a hard thing to test from the outside. (And it might depend on whether the email the email was queued at the time or came in later.)

Written on 20 October 2007.
« Some notes on booting single user in x86 Solaris 10
How I got a corrupted metadb replica that paniced Solaris 10 x86 »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Oct 20 22:41:44 2007
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.