A Python code structure problem: exception handling with flags

July 31, 2013

Here's a Python (2.x) code structure puzzle that I don't have a good answer for yet, except that maybe the answer is that my overall design is a bad fit for what I'm doing. To start with, suppose that you have a multi-level, multi-step process of processing lines from an input file. Any number of things can go wrong during the processing; when it does, you need to bubble this information up to the top level but keep on going to process the next line (if only so you can report all of the errors in the file in one pass). The obvious fit for this is to have errors communicated by raising exceptions which are then trapped at the top level.

Now let's suppose there are several different sorts of errors and you want to treat some of them specially based on command line flags. For example normally all errors are fatal and show error messages, but some can be suppressed entirely with a flag (they just cause the record to be silently skipped) and some can be made into warnings. How do you structure this in the code?

My first version looked something like this:

   data = crunch(line)
except A, e:
   commit = False
except B, e:
   if not option.skipempty:
      commit = False
except C, e:
   if not option.skipdups:
      commit = False

All of the duplication here made me unhappy because it obscured the actual logic and makes it easy for one exception to drift out of sync with the handling for the others. I can aggregate everything together with 'except (A, B, C), e:' but then the question is how to write the single exception handler so that it's both clean and does everything necessary; so far I've thought of two approaches. The first approach is to use isinstance() on e to tell what sort of exception we have and then write out the conditions in if's, except that trying to do that makes for ugly long conditions.

(I started to write out the example above and basically exploded in irritation when I got to the commit logic, which I decided was a bad sign. It also looked like the result would be very hard to read, which means that errors would be easy to add.)

The second solution I've come up with is to add attributes to each exception class, call them report and nocommit. Then at the start of the code we do:

if options.skipempty:
   B.nocommit = False
if options.skipdups:
   C.report = False
   C.nocommit = False

In the main code we do:

except (A, B, C), e:
   if e.report:
   if e.nocommit:
      commit = False

This avoids both duplication and lack of clarity at the expense of, well, kind of being a hack.

(You can also code a variant of this where report and nocommit are functions that are passed the options object; this puts all of the 'what turns this off' logic into the exceptions themselves instead of reaching into the exception classes to (re)set attributes. That might be somewhat cleaner although it's more verbose.)

Given that all of the options have drawbacks I feel that there ought to be a clever Python code design trick that I'm missing here.

Comments on this page:

From at 2013-07-31 05:32:51:

I'm not sure why you consider extending the extensions with attributes a hack, but for sure it's adding much stuff in various places, so I agree it could be not the prettiest way.

May I suggest adding exception classes to a dict or set ? Then you do something like this:

nocommit = set([IOError, FooException, BarError])
report = set([FooException])

except Exception, e:
  if type(e) in nocommit:
    commit = False
  if type(e) in report:

The drawback is that this doesn't work for exception subtypes.

-- dozzie

Written on 31 July 2013.
« Phish spam and outside events
The problem with a custom laptop environment: designing it »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Jul 31 00:38:13 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.