The paucity of generally useful HTTP error codes
One of the things that I didn't appreciate until I really looked at
HTTP error codes is how few generally useful
ones there are. To start with, we can divide HTTP error codes into two
categories: specific technical failings and general errors. Specific
technical failings are things like an
Accept: header that the server
can't satisfy. There's a bunch of 4xx errors for these cases (and a
few 5xx errors), but they aren't useful in general since you're only
supposed to generate them in specific technical circumstances.
Once you get into the officially specified general errors, though, there simply aren't that many: 403 Forbidden, 404 Not Found, 410 Gone, 429 Too Many Requests, and maybe 451 Unavailable For Legal Reasons (if you accept Internet drafts) and 400 Bad Request (if you stretch it). On the server error side, 500 Internal Error and 501 Not Implemented are basically it. Of the 4xx errors, only 403 and 404 are really general.
(It's striking how many unofficial HTTP error codes there are in the Wikipedia list. Apparently a lot of people have found the current set inadequate.)
This limited set of error responses means that a web application can't really tell clients very much about what went wrong using error codes alone (at least officially assigned ones). Consider, for instance, a web application that both has access-restricted content and blocks certain clients from some or all content. HTTP error codes alone provide no real way to distinguish between 'you can't have this content because you aren't properly authenticated' and 'you can't have this content because I think you're a robot and robots shouldn't be asking for this' (especially if the web app also has rate limiting and so uses 429).
This has wound up tied into my feeling that specific HTTP errors may not matter that much. If the available HTTP error codes are too limited to really communicate what you mean to the client, your choice of what specific error code you use from the limited general-use set is not necessarily very important.
Sidebar: technical failings versus general errors
I've realized that I draw a big, personal distinction between these two that doesn't necessarily exist. I consider technical failings to be the job of the web server (and the framework if any) to worry about and I basically ignore them when writing an application. The errors I care about are general errors.
Thus I need to clarify and effectively walk back some stuff I said when I asked whether specific HTTP error codes mattered. Getting the error code right for specific technical failings does matter (at least in theory). I was intending to focus on general, application level errors but both didn't make that clear and didn't appreciate just how many 4xx errors there are for technical failings until I'd looked at a list.