Chris's Wiki :: blog/tech/SpecMustVsShould

MUST versus SHOULD in your specifications

August 29, 2009

Something I've been mulling over lately is the the effects of A rule for Internet software on specifications and how strong you make requirements in them.

To put it one way: a 'MUST' in a specification makes a handy club to beat people with, but perhaps they are best reserved for things that are absolutely required in order to make the protocol work, because in practice that is what is going to happen anyways.

(Note that a MUST often does not make an effective club, because if it doesn't break things the people who got it wrong rarely care about it. This is the world of 'technical' violations of standards, as in 'technically we don't comply with ...'.)

Given that people screw things up, sooner or later someone is going to write software that disobeys every MUST in your specification (maybe not all at once, though). Some of these violations will cause their software to not work, and they will get fixed. Some of them will not, and generally those violations will then live on in the software. At this point, the only thing that those MUSTs in your specification are doing is giving people an excuse to lecture the author (and sometimes the users) of the software, and perhaps to feel smug when they do things that make it not work.

This gets stronger when you are talking about user visible behavior, because sooner or later people will disagree with you about the right thing to do (sometimes they will even be right in practice). At this point, all of these MUSTs are pointless, and should be SHOULDs all the way through.

(If you have a 'how things are presented to users' MUST that really is a MUST, for example because it's necessary to preserve security properties, you have a serious problem, because sooner or later that MUST will be violated.)

I don't particularly like this conclusion; in some ways I'm a fairly strong protocol purist (which makes me an asshole). But I think that it's inescapable in the real world, and certainly I think this is how it's played out repeatedly in practice with things like SMTP.

Note that I'm talking about, essentially, protocol specifications here, ones that involve multiple computer parties that actively interact with each other. ESMTP is a protocol specification, but HTML is not. I'm not sure what MUSTs in HTML-like specifications do in practice; possibly their only real use is making it so that people can't trivially claim compliance with your specification.

(All of this is similar to the IETF usage of MUST and SHOULD, but not quite the same. I think that the IETF would have people use MUST in places that I would wince, hold my nose, and suggest SHOULD.)

Comments on this page:

From 78.35.25.18 at 2009-08-29 08:23:25:

At this point, all of these MUSTs are pointless, and should be SHOULDs all the way through.

Arrrrrgh, noooooo! You SHOULD avoid “SHOULD”. Use MAYs as much as possible. If you can’t, make it a MUST. But if there are no objective criteria for compliance, as a last resort, leave it a SHOULD. Because in practice, all SHOULDs become MAYs or MUSTs.

—Aristotle Pagaltzis

By cks at 2009-08-29 13:14:18:

I am not sure that I understand the IETF usage, but here is how I see things:

MAY: do this if you want to.

SHOULD: do this if you can.

MUST: do this or your program doesn't work with anyone else.

Consider the ESMTP specification. Currently it says that the SMTP client MUST ensure that it EHLOs with a valid host/domain name. This is clearly a bad MUST, as nothing in the protocol breaks if you violate it (and people do all the time). You could make it a MAY, but this doesn't give enough encouragement to client implementors to do it, so I would make it a SHOULD.

From 78.35.25.18 at 2009-08-30 04:48:22:

Yes, there are bad uses of MUST and SHOULD, even of MAY, in IETF RFCs. The IETF is not uniform; they provide an infrastructure umbrella for working groups, but each RFC could be written by a different group. Groups are assisted by an IETF person or two or three, but those people rarely do much of the technical work, just the procedural wrangling. At various points, the in-progress RFCs are subjected to critique by IETF reviewer groups; the IETF also has certain standards and conventions (such as RFC 2119) for specs. But overall, it’s a very mixed effort, so the use of 2119 terms varies across specs, and instances of such terms within specs may well be inconsistent or unwise uses.

That’s the reason I felt compelled to write down what I learned about their use during my participation in such a working group, too.

Personally, not knowing the specifics, at first glance, I would use a MAY in the EHLO case and include an explanation in the accompanying informal text for why implementers should choose to provide proper data.

The central criterion for when to use which is always interoperability. SHOULD should be reserved for cases where an implementation continues to interoperate in a strictly technical sense, but the violation severely degrades usefulness, eg. an HTTP server that always puts the current time in the Last-Modified header and generates a new hash for the ETag header on every request: there’s no criterion by which a validator could reliably detect and fault this behaviour, so you can’t quite make it a MUST, even though it completely defeats all the HTTP provisions for intermediaries.

Basically, SHOULD is supposed to mean “you’re going to cause severe degradation if you break this rule, therefore, while we’ll let you break it, you should think really really hard before you do so – and if things work really badly afterwards, don’t go crying to anyone”.

And as I said, in practice, all SHOULDs tend to be either universally ignored (turn into MAYs) or universally implemented (turn into MUSTs). So using a lot of SHOULDs is a sign that the spec writers couldn’t decide what to do so offloaded their job to the implementers. SHOULD is the standards body equivalent of “let’s make it a preference”.

—Aristotle Pagaltzis

Written on 29 August 2009.

«	ZFS changes filesystem device numbers on reboot

Some more thinking about requirements in specifications