My view on accepting bounces and replies to your email

November 11, 2016

This topic came up in comments on yesterday's entry about Yahoo Groups not accepting bounces of their email, so I'm going to make it an entry and elaborate on my views here.

My views on accepting bounces are pretty simple: if you send out email using anything but the SMTP null sender, you have a responsibility to accept bounces of it back. In fact, not just bounces alone; bounces and replies, because some of the recipients will have out-of-office and other auto-responders (whether implemented on the server or in their IMAP client, and yes I've seen both). Since you have to accept replies and replies may be in arbitrary formats and come from random addresses that are the result of forwarding and other things, in practice you must accept all email to any address that has recently sent out email.

(We can have a rousing debate over whether you 'must' accept email merely to the MAIL FROM of outgoing email or also to the From: or Reply-To: of the email. My pragmatic answer is that there exist systems that will send replies to the latter, so yes, you should.)

By 'accept' I mean merely that your SMTP receiver must accept the message during a SMTP transaction. Technically you may immediately drop the message in /dev/null if you want; your obligation is merely to insure that the mail systems generating bounces and replies are not stuck with either the bounces (as they would be if you don't accept or reject them outright) or bounces-of-replies (as they would be if you rejected replies).

(Throwing bounces away locally is a sign of a spam operation, since you're ignoring delivery failures and will therefor continue grimly trying to email addresses that are simply not working. But that's different from irritating other mail system operators.)

You have no obligation to accept bounces, replies, or even general email from the null sender address to addresses that have not sent email, or that do not send email that will ever leave your domain. In particular, people with sender address verification systems that assume that all addresses will accept email from the null sender are doing it wrong.

(Not that sender address verification is a good idea in general, but there are more worse and less worse ways of implementing it.)

Since modern mailing list systems individualize the MAIL FROM of every outgoing message to every recipient, they have a built in way of knowing what local email addresses have recently sent email and thus should accept it back. Or they can just accept everything if they're going to throw it all away anyways.

When receiving replies I feel that it is legitimate to respect the DMARC policies of the MAIL FROM, even if this causes you to reject some incoming messages. I will wave my hands about rejecting things at SMTP time that are properly identified as spam. In the specific case of mailing list software and VERP, I think you should accept everything since it's just going to automation.

On a purely pragmatic level, I've written before about how not accepting bounces make you look bad and how broken bounce addresses don't fool anyone any more. The pragmatics are pretty clear: if a random new sending domain doesn't accept bounces and (auto-)replies to their email, the odds are very good that they're a spammer. It's not a sure thing, but it is a strong smell and you can expect people to react to it.

(And it's not as if it's particularly difficult or expensive to set up a simple email server that just throws away everything it receives. I even have something that can do this sitting around and I'm pretty sure you could run a high-capacity install of it on the smallest, cheapest instance AWS offers.)

Comments on this page:

By Aneurin Price at 2016-11-12 10:37:39:

Thanks for going into more depth on this. I'd like to return the favour - hopefully without sounding combative, but if I fail at that then I apologise; I'm not a great writer.

First a quick note: the systems I administer do accept bounces (subject to the same rules as any other mail, which in practice means they can be rejected at submission time for numerous reasons, including "SpamAssassin says 'lol no way'"). I mention this to point out that I'm not just scrambling to come up with justifications for something I'm doing.

if you send out email using anything but the SMTP null sender, you have a responsibility to accept bounces of it back

I don't think this can be reasonably treated as axiomatic; it needs some justification.

In some sense, bounces can be broken down into two categories: bounces that are internal to some system - sent from an MTA you control/trust back to a client you control, and those that come from an external MTA.

The first category is straightforward and is the standard way that your users know their mail couldn't be delivered: your outgoing MTA by design relays from your users to anywhere, so it accepts anything from them without any knowledge of the destination (with the usual caveats about sending restrictions/filtering rules/whatever). Thus if it turns out to be undeliverable then it bounces it back, and it know it's always safe to do so.

The second category is the thorny one: mail that's sent by one system, accepted by an unrelated system, and subsequently rejected by a third unrelated system. Note that there must be three systems involved for this to be legitimate; it is never acceptable for an MTA to accept incoming mail and then later unilaterally decide to bounce it.

So in what cases can this second category be legitimate? Typically there should only be two systems involved in the delivery process: the sender and the receiver (they probably have relaying MTAs internally, but logically that's part of the same system). Since a valid external bounce can only exist when there are three, this boils down to "in what cases is it legitimate for three systems to be involved?". There aren't many; in fact, I can only think of one, which is the case that's caused your problem: allowing users to relay mail elsewhere.

In this case, your MTA-in-the-middle is in a bad spot. All it can really do is accept mails in good faith based on the criteria it would normally use, then attempt to deliver them in good faith based on the rule that the user has set. If something goes wrong, it's the one stuck with the problem. Typically it's not okay to bounce: it's overwhelmingly more likely that the mail was spam than that this is a useful NDR. It sounds like you're verifying the sender and only generating a bounce when you're confident that you're bouncing to the right place, and just silently dropping any other undeliverable messages. Assuming that's the case, congratulations: you're in the 0.1% that's both in a position to send the one type of legitimate external bounce, and not sending any others. If your bounce can't be delivered, well it sucks that your sender verification was all for nothing, but ultimately it just means you have to treat it like unverified mail and just drop it. That's probably the more common case anyway - I presume that the majority of the incoming mail that you have to bounce does not come with a way to verify the sender?

Throwing bounces away locally is a sign of a spam operation, since you're ignoring delivery failures

This is very definitely not the case. In nearly all circumstances, a failure to deliver should not result in a bounce; it should be immediately rejected.

Rejecting bounces doesn't mean you're ignoring delivery failures, it means you're ignoring delivery failures in the one vanishingly small case when a bounce is the legitimate way to report a delivery failure. On the other hand, it means you're rejecting a great deal of spam and backscatter. I was being generous with the 0.1%, and I also wasn't taking into account the relative volumes of mail between spammers and honest senders. I think a realistic conservative estimate of the ratios involved would be that you're probably losing about one legit NDR per hundred thousand spam mails dropped. Adding another order of magnitude would not sound implausible to me.

If Yahoo can verify that the bounce is a real response to a mail they know they've sent, then yes, they should be accepting it rather than silently losing the NDR. Not doing so doesn't even remotely imply to me that they're doing it because they're enabling spammers though, just that they've deemed it not worth the cost of handling this one case amongst all the others. If this is something that they're doing specifically for Groups and not other Yahoo mail, then I'd wager [a very small amount] that the actual reason is to block auto-responses (which are never wanted by a mailing list, but numerous), with the occasional lost NDR being unfortunate collateral damage. Either way, it doesn't seem at all like spammer behaviour to me. It doesn't seem like a very good idea, but it doesn't look in any way nefarious. (And don't forget Hanlon's razor; people do far worse things all the time and it's usually because they haven't thought it through rather than because they're acting in bad faith.)

I have even more to say on this topic, but this comment has already got out of hand and hopefully it's reasonably coherent as it stands.

By cks at 2016-11-12 14:18:44:

One quick clarification; when I wrote:

Throwing bounces away locally is a sign of a spam operation, since you're ignoring delivery failures [...]

Here I meant this in the context of being the sender of the email, ie you as the sender accepting bounces back but then dumping them in /dev/null instead of doing anything with them, not in the context of being the receiver. If you as the sender ignore bounces that you get back, what you wind up doing is continuing to blindly send to dead or non-functional addresses. People will notice that, just as they will notice that you keep trying to RCPT TO to addresses that have been rejecting you for years.

Written on 11 November 2016.
« Yahoo Groups slides further down the spam source slope
Modern shells and running shell scripts while seteuid »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Nov 11 22:12:23 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.