A SMTP implementor's conundrum

December 13, 2006

Today I ran across a good example of the sort of engineering conundrum that bedevils people who implement things like SMTP clients:

In a multiline SMTP reply, which reply code should you use if different lines of the reply have different codes?

That is, in a regrettably non-hypothetical example, if you send a server 'MAIL FROM:<something>' and get back:

250-There's a problem.
454 please try later

Should your client go on and send a RCPT TO, go away to try again later, or just run screaming?

(Sending such a multiline reply is not spec compliant, but our core mailer is held together by bailing wire and chewing gum, so these things surface every so often.)

RFC 2821 is silent on this, unsurprisingly. Our usual MTA and all the code I've had a hand in has used the last line's reply code, and I've been assuming that that's how everyone operated. But today I found out that Exim appears to use the first line's reply code instead, and so thinks that its MAIL FROM was accepted.

(Then when Exim goes on to send a RCPT TO, the server sees it as an out of phase command, which is a 5xx hard error and Exim bounces the message. I think they were all spam messages, so I'm not too broken up about this.)

I think our choice of using the last line's reply code makes more sense (and is more likely to be the server's intention). But I can't exactly blame Exim for its decision, and I have to admit that I'm biased.


Comments on this page:

By Dan.Astoorian at 2006-12-13 10:23:57:

In a multiline SMTP reply, which reply code should you use if different lines of the reply have different codes?

As a matter of principle, IMHO, you should use neither code if you detect this condition. You should conclude that the server you're talking to is not speaking SMTP after all or has lost its mind, terminate the operation and the connection, and do whatever you would have done (such as falling back to the next MX host) if, e.g., the server had closed the connection without warning.

However, practically speaking, one should not expected to detect this condition, let alone anticipate it, and one may reasonably use whichever code is programmatically expedient, and blame the server for any consequences that result. It's the client's job to comply with the standard, not to enforce it.

(Sending such a multiline reply is not spec compliant, but our core mailer is held together by bailing wire and chewing gum, so these things surface every so often.)

Although that explains the behaviour, it does not excuse it. (I know whereof I speak--I used to maintain that particular wad of chewing gum, before cks replaced the sticky tape with bailing wire.) As it usually does, it comes down to a fairly straightforward choice: fix (or at least improve) the behaviour, or tolerate the consequences. Figuring out what the SMTP client "should" do in the face of insane server behaviour amounts to blaming the victim.

--Dan

By DanielMartin at 2006-12-13 15:06:41:

See, if I were writing a truly naive client, I would accept the first line's status code, and then accept the second line as the response to my RCPT TO, which would, in this case, result in vaguely acceptable behavior. (try again later)

I wouldn't be surprised if various spamming programs shared this behavior. In fact, Googling for "RCPT TO" in perl scripts leads me to believe that this pattern of coding is pretty common. Most people don't write code assuming multi-line replies.

By cks at 2006-12-18 14:34:44:

I think I will say a heartfelt 'augh' at that code. I had no idea anyone would do that (especially in what looks like real and serious code), although I suppose I really shouldn't be too surprised.

(In this particular case we were actually generating multiple continued lines, all with 250- at the front, so that code still wouldn't get it 'right'.)

I suppose one good thing about ESTMP is that anyone who tries to use it can't help but be aware of multiline replies, since multiline replies from EHLO are routine.

From 68.215.50.125 at 2007-04-26 12:18:21:

Chris,

I wish I would of found your blog LAST WEEK!! I could of used you to second the issue regarding mixed codes in multiline responses. Like you, I was surprise to find EXIM reading the first code, not the last.

Like you, the RFC821 and RFC2821, both provides an interpretation that the LAST CODE is what is important and that all systems operated in the same manner.

In our case, we wanted to use 150- lines to help keep MUA clients alive during long mail processing jobs. For example, this works with most MTA and MUAs:

  C: DATA
  S: 354 Start sending email
  C: [upload data]
  S: 150-Please Wait... (50 secs)
  S: 150-Please Wait... (100 secs)
  S: 250 Message Accepted

Unfortunately, as we found out in the IETF SMTP working group discussion the update to RFC 2821BIS, it is not only EXIM, but QMAIL and other servers out there that have the same first code reading.

Thus it was deemed more important to correct the specification with new clarifying text that says all reply codes in multi-line response MUST be the same.

You should probably join the IETF-SMTP working group to get your input on the specification before it goes to draft. But you better hurry.

Here is a link to the current RFC2821bis draft:

http://tools.ietf.org/html/draft-klensin-rfc2821bis-03

--- Hector Santos, CTO Santronics Software, Inc. http://www.santronics.com

By cks at 2007-04-27 09:05:19:

I actually think the IETF SMTP working group is making the right choice from both a standards perspective and a pragmatic perspective.

Clearly the existing specification is ambiguous for clients, so either choice is valid and there are clients that do both. In this case it is wrong to change the specification so that half of the clients are now suddenly wrong; it is better to formally insist that both can be right, especially if it was the intent all along that multi-line responses need to use the same response code on each line.

From a pragmatic perspective, there are popular clients that do both out in the field now. Even if a new SMTP RFC was issued tomorrow that mandated one or the other, it would be years before even the majority of those clients were updated, so the most useful thing to tell server implementors is 'you must make all of the response codes the same, or you will see explosions'.

Written on 13 December 2006.
« A limitation of OpenBSD bridging NAT firewalls
An example of Unix's slow fossilization »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Dec 13 01:06:35 2006
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.