Why Google's handling of multiple domains on inbound messages is okay

November 17, 2012

It started with a tweet by @xlerb (Jeb Davis):

Today I learned that Google thinks they can unilaterally redefine SMTP: <link> (warning: gratuitously shouty forum comments)

To summarize the link: Google is the MX target for all sorts of domains, due to various services they offer. Google's MX servers will now only accept destination addresses in a single domain per transaction; if you try to RCPT TO to addresses at multiple (Google-hosted) domains in the same transaction, all but the first domain will get 4xx temporary failures.

I'm not particularly fond of Google's handling of email, but as I tweeted I come down on Google's side here. First off, this can't possibly be called 'redefining SMTP'. Mail servers have always been allowed to temporarily defer some recipients for any reason whatsoever, including random software limitations, their own convenience, and obscure internal policies. Anyone who expects all recipients to always be accepted on the first delivery attempt has not been paying attention to the modern Internet mail environment for years; many, many systems behave otherwise (ours included). The only vaguely novel thing Google is doing is that they are being clear about why addresses are getting temporary failures.

(It would be redefining SMTP if Google was giving 5xx permanent failures in this case and telling everyone to fix their software to not do this, but not even Google is that stupid.)

Second, there is an excellent reason why Google might want to do this; it is my old friend the lack of partial success for message delivery. If different Google-hosted domains can have different policies on what message contents can be sent to them (perhaps Google allows the domain owners to control this), Google needs to make sure that it's never in a situation where some recipient domains would accept the message but others would refuse it. Giving each domain its own MX IP address (or set of them) is not exactly a scalable solution (not at Google's scale), so Google has to do it the other way; they can only ever accept a single domain per transaction, so only a single domain's policy will apply to the message contents.

Finally, my view is that any significant mailing list operation that's having problems about this is probably doing things wrong. For a start, any mailing list using VERP will not be affected by this, because with VERP each transaction has only a single recipient. And you should really, really be using VERP and automated bounce handling if you're running a mailing list of any appreciable size.

(Note that in theory, ordinary people can run into this routinely; all you need to be doing is having a conversation with several people who are all hosted through Google but at different domains. Depending on how fast your mail system does retries, some of the people in the conversation may get messages much slower than others. In practice, who knows what special magic Google is doing.)

All of this is something that goes well beyond what Google is doing right now. Every mail server that wants to make accept/reject decisions based on the both the message contents and the destination addresses (or domains) faces this issue, and there are no better solutions than what Google is doing. If you want to allow people or hosted domains to reject during SMTP (and you do), and you want to give them some control over what gets rejected, you're going to wind up doing the same thing.

(And you should not feel particularly broken up about it. Batching multiple addresses with different destination domains together into a single transaction when they all MX to the same thing is an optimization, not a fundamental feature of SMTP. It just happens to be a common optimization in mailers, partly because it's cool in a way that attracts programmers like bees to honey.)

Written on 17 November 2012.
« Why DTrace does not attract people to Solaris very often
Linux is less divergent than commercial Unixes used to be »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Nov 17 01:09:13 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.