Wandering Thoughts archives

2011-08-26

Why we don't use a trouble ticketing system

In an aside in this entry I mentioned that we don't use a trouble ticketing system here, and a commentator asked the obvious question of why not. The short answer is that we don't think moving from our current approach to a trouble ticketing system would be useful enough to justify the amount of extra work and annoyance it would take.

For the long answer, I am going to make up a hierarchy of approaches to tracking your work, especially the work that people ask you to do:

  • level 0: people make direct contact with individual sysadmins and everyone is working more or less independently. If a sysadmin isn't available, most things they were working on get dropped or put on hold.

  • level 1: you have a central mailing list or other point for all email with people; they send email to the mailing list to ask you for things and sysadmins copy their replies to the mailing list.

  • level 2: you have an actual trouble ticketing system with automation, a website, status summaries for tickets, reporting, and all of that.

Moving from level 0 to level 1 is a big deal. It changes your work life in a very positive way; it means that everyone can stay informed about what's going on and any sysadmin can pick up some particular task from another sysadmin. A level 1 system is not perfect for the same reason that our old account request system was not perfect; reconstructing the state of any particular request can take trawling through your email archive in order to find all of the messages, and it's hard to get an overview of all of the open requests.

Moving from level 1 to level 2 has benefits but in many environments it's less of a sea change than moving from level 0 to level 1, and it comes with overhead; an actual trouble ticketing system is a form of bureaucracy and dealing with it is invariably more work than sending a quick freeform email message. My feeling is that a level 2 system is justified when you have enough request volume that you need it to keep track of things. I can think of two warning signs of this; first, when individual sysadmins start ignoring most of the mail from the central mailing list in order to survive the volume, and second, when you frequently have more pending requests than you have sysadmins (then a ticketing system helps you to easily find the next thing to work on).

(Your manager may also demand a trouble ticketing system in order to make it convenient to generate various metrics.)

Right now, we have a level 1 one style system with a central mailing list and a relatively low request volume. So for us, moving to a ticketing system would add bureaucracy (and another software system to run) without having particularly compelling benefits; we simply aren't active enough to make it a necessity instead of something to play with.

sysadmin/WhyNotTTSystem written at 16:11:10; Add Comment

The problem for WSGI middleware

I've recently come to a (belated) realization about WSGI.

Looked at from the right perspective, WSGI is a simple protocol hiding inside a complex one (or you could say that it's designed to make simple things relatively simple and complex things possible). Almost all WSGI applications are written to the simple protocol and I think that many server implementations effectively are as well, especially ones written in specialized circumstances for private use.

This is great for application writers; most application writers only need to do simple things that fit nicely into the simple protocol. It's okay for server implementors who have relatively simple needs, since they can mostly copy from the sample PEP implementation and forget about it.

(It's not so great for server implementors who really do need the complex stuff; not only do they have to implement it, but they may find all sorts of other people's WSGI code that chokes on it.)

But it's terrible for middleware developers, because they never get to deal with only the simple subset of WSGI. At least in theory, middleware has to support the full complexity of the WSGI protocol including all of the odd and basically never used bits, because it might someday be used in a sufficiently complex and perverse environment. Then much of that code sits there unused almost all of the time, because after all most of the time WSGI is used in simple ways. The frequent result is that either people don't write middleware at all or they write incomplete middleware that only works in some WSGI environments. Sometimes they write incomplete middleware knowingly and deliberately, because it's all they need, and sometimes they write incomplete middleware without realizing it.

(PJ Eby has recently said that basically all WSGI middleware libraries he's looked at have bugs handling sufficiently perverse but legal WSGI environments.)

(I've written about this issue before from a somewhat different perspective in WSGIGoodBad. There I was focused more on the intrinsic complexity that middleware has even when used in the simple WSGI environment, partly because at the time I didn't understand how complex WSGI could get. WSGI middleware effectively has two sorts of complexity to deal with, but that's another entry.)

Sidebar: where I think middleware complexity lives

My intuition is that there are three general cases for middleware, in order of increasing complexity. The simplest case is middleware that either intercepts the request, acting as an application, or passes it down unmodified. The middle case is middleware that modifies the request before passing it down to the next layer. The most complex and problematic case is middleware that must modify the response as it comes back, especially if it has to do something before the request is passed down.

Oh, and only request headers are particularly easy to modify. If you need to modify the request body for a POST request, well, I think you have a bunch of pain coming.

python/WSGIMiddlewareProblem written at 01:17:56; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.