Wandering Thoughts archives

A problem with signals in Python

As a followup to what I wrote yesterday about prompt signal handling in programs, it's worthwhile to point out a little problem in Python with signal handling.

As I've noted before, Python normally turns many signals into exceptions. It turns out that this has an important consequence: it delays processing of signals, because Python only processes signal exceptions in the interpreter, ie when you're running Python bytecodes.

(Contrary to the documentation, sys.setcheckinterval does not appear to control how often signal handlers are run; the CPython code that handles signals overrides the check interval value to force an immediate check.)

This matters because Python can't abort long operations being done in low-level modules, like looking up hostnames and making network connections. When you hit ^C during one of these, all that happens is that Python notes that it's got a pending signal; only when the operation finishes and control returns to the Python interpreter does the signal take effect.

(Python can't abort the operation because there's no way to clean up whatever internal state C code may have (especially in libraries). Just blowing away the operation without cleaning up this state may leave data structures corrupt and cause all sorts of problems later. This isn't specifically a Python issue; portable code can't really do very much more in signal handlers than set a flag.)

If you don't need to catch signals at all, the best way to fix this problem is to set them to SIG_DFL using the recipe in my earlier entry. If you do need to catch the signals to clean stuff up, unfortunately there's no good way out.

Embarrassingly, I've been caught out by this in several of the little tools I use because it's always seemed like too much hassle to put in the signal dance, and after all I didn't care if ^C got me an exception puke instead of a quiet death. It's tempting to make a module that does it all for me, so I can just put 'import unixy' or the like at the start of my little programs and have it just work right.

(Well, it'd take a bit more than just an import unless I put the module somewhere on the standard module search path. Details, details.)