Three ways to get tracebacks in your CGI Python application

January 22, 2010

There are at least four ways to get Python tracebacks in your CGI or web-app in case something goes wrong; the easy but wrong way, the more or less right but inconvenient way, the flawed way, and the good way.

The easy but wrong way is 'import cgitb; cgitb.enable()'. The reason this is the wrong way (except when you yourself are developing your program) is that this dumps error tracebacks on the users of your application. Ignoring any information disclosure issues, reporting errors to the users is both wrong and pointless; the odds that users will actually bother to save the traceback and report it to you are somewhere between low and nil.

The more or less right way is to use the cgitb module to log tracebacks to files in some directory somewhere, and not display the traceback to your users. This has two drawbacks; first, your users get at best a very minimal error message, and second you have to monitor the directory to check for problems.

(It may be helpful to know that the cgitb module still produces a very brief report to standard output even if it is set to not display the traceback itself. If you use HTML format tracebacks, this report is in HTML, cleverly formatted to be valid either as HTML or the entire output of a CGI program. If you use plain text tracebacks, this report is in plain text and probably won't display in people's browsers, which may be considered a feature.)

The flawed way is to not do anything special, so that you CGI just writes a regular Python traceback to standard error and thus to your web server's error log. The drawback of this approach is that it makes it hard to pick things out of the error log. While web servers are getting better about making sure every line of the error log has basic information (such as timestamp and request source), your messages won't have things like the program that produced them. Among other issues, this can make it hard to find out that your program is having problems.

The good way is to catch the traceback and reformat it to have a distinctive structure. What I do is prefix each line of the traceback with '<program>: T: ', and surround the whole traceback with a start and an end line to make it easier to pick out the whole thing. You can set this up as the system exception hook, making this approach just as easy to use as the cgitb module.

(Having just reminded myself how simple the good way is to use, I think I should go plug it into some of our internal CGIs (which are currently using the more or less right way).)

Sidebar: my stderrtb module code

Here is the entire code I use for this in DWiki:

import sys
from traceback import format_exception

def stderr(msg):
    sys.stderr.write("%s: %s\n" % (sys.argv[0], msg))

def print_except(t, inf, tr):
    stderr("Python internal error:")
    for e in format_exception(t, inf, tr):
        lines = e.split("\n")
        for l in lines:
            if not l:
                continue
            stderr("T: %s" % l)
    stderr("-- traceback finished")

def enable():
    sys.excepthook = print_except

Call the enable() routine to turn this on. Note that this needs no support from your application; it works just like the cgitb module does. You can reuse print_except if you need more sophisticated error handling and recovery in your application.


Comments on this page:

By Dan.Astoorian at 2010-01-22 11:06:13:

I take a slightly different approach: if the traceback occurs in a session which has not logged into the site, or has logged in as a normal user, the traceback, as well as other useful information about the session in which it occurred, is written to an error log directory (by my code, not by cgitb); however, if the session is logged in as a developer (i.e., a user with the "debug" privilege), the traceback is displayed to the browser instead.

This means that while developing, I don't need to go open a new window with the error log every time I make a typo, and moreover, the error log directory doesn't get cluttered while I'm testing the site, so I know that if anything shows up there, it's definitely something that requires attention.

I don't agree that having to monitor a traceback directory is a drawback; personally, I find it considerably easier than having to tease the same information out of the web server logs.

Written on 22 January 2010.
« The argument for not managing systems via packages
A Python safety tip: Do notice that things can throw exceptions »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jan 22 01:00:04 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.