The problem for WSGI middleware

August 26, 2011

I've recently come to a (belated) realization about WSGI.

Looked at from the right perspective, WSGI is a simple protocol hiding inside a complex one (or you could say that it's designed to make simple things relatively simple and complex things possible). Almost all WSGI applications are written to the simple protocol and I think that many server implementations effectively are as well, especially ones written in specialized circumstances for private use.

This is great for application writers; most application writers only need to do simple things that fit nicely into the simple protocol. It's okay for server implementors who have relatively simple needs, since they can mostly copy from the sample PEP implementation and forget about it.

(It's not so great for server implementors who really do need the complex stuff; not only do they have to implement it, but they may find all sorts of other people's WSGI code that chokes on it.)

But it's terrible for middleware developers, because they never get to deal with only the simple subset of WSGI. At least in theory, middleware has to support the full complexity of the WSGI protocol including all of the odd and basically never used bits, because it might someday be used in a sufficiently complex and perverse environment. Then much of that code sits there unused almost all of the time, because after all most of the time WSGI is used in simple ways. The frequent result is that either people don't write middleware at all or they write incomplete middleware that only works in some WSGI environments. Sometimes they write incomplete middleware knowingly and deliberately, because it's all they need, and sometimes they write incomplete middleware without realizing it.

(PJ Eby has recently said that basically all WSGI middleware libraries he's looked at have bugs handling sufficiently perverse but legal WSGI environments.)

(I've written about this issue before from a somewhat different perspective in WSGIGoodBad. There I was focused more on the intrinsic complexity that middleware has even when used in the simple WSGI environment, partly because at the time I didn't understand how complex WSGI could get. WSGI middleware effectively has two sorts of complexity to deal with, but that's another entry.)

Sidebar: where I think middleware complexity lives

My intuition is that there are three general cases for middleware, in order of increasing complexity. The simplest case is middleware that either intercepts the request, acting as an application, or passes it down unmodified. The middle case is middleware that modifies the request before passing it down to the next layer. The most complex and problematic case is middleware that must modify the response as it comes back, especially if it has to do something before the request is passed down.

Oh, and only request headers are particularly easy to modify. If you need to modify the request body for a POST request, well, I think you have a bunch of pain coming.

Written on 26 August 2011.
« Trouble ticketing systems and the future
Why we don't use a trouble ticketing system »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Aug 26 01:17:56 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.