Wandering Thoughts archives

A Python safety tip: Do notice that things can throw exceptions

Recently, I discovered and fixed a long-standing bug in DWiki. The bug itself was simple, but typical of a certain kind of error that I keep making in Python: I had forgotten that the Cookie module could throw an exception, so I didn't catch it.

(Well, my general error code logged it, but I mean DWiki itself didn't catch it and do anything sensible; in this case, either ignoring the broken HTTP cookie header or perhaps reporting a security error.)

This bug lingered for so long because the Cookie module more or less only throws errors for invalidly formatted cookies, and invalidly formed cookies are often very rare. Most browsers make sure to only send valid cookies themselves and are very careful about not sending you other people's cookies, so under many circumstances it can take either a badly broken client or someone deliberately trying to attack you before you see a badly formed cookie. The net result is that you could run a web application for years before this particular code path gets triggered.

(This is what happened to DWiki, and I'm not sure if it was an attack or just a really, really bad client. Possibly both.)

I have no real excuse for this; the Cookie module's documentation certainly covered the CookieError exception at the time that I wrote the relevant DWiki code. I just didn't read that bit of the documentation. (The current module documentation contains a more prominent caution about this, perhaps caused by other people also skipping over the previous version.)

This sort of mistake is almost inevitable in any language that doesn't force you to explicitly code error handlers (either by forcing you to explicitly handle exceptions, or explicitly including a status return value and making you do something with it); Python at least insures that the error is handled, one way or another. Doing better is probably the domain of code analysis tools like pylint, although detecting this sort of thing might take fairly serious analysis work unless people start annotating code and modules with information about what exceptions they raise.

(I don't believe that test driven development would have caught this error, since I doubt I would have thought to include a test with a corrupted cookie. If I had been thinking about that possibility in the first place, I would have found the exception in the documentation.)