== What the members of a Unicode conversion error object are

The [[codecs module http://docs.python.org/library/codecs.html]]'s
((register_error)) function has a rather scanty description of what
error handlers are called with. Since I have just dug into this,
here is what the members are for at least UnicodeDecodeError:

|_. _object_		| The string that is being decoded to Unicode.
| _encoding_		| The encoding that the string is (theoretically) in.
| _start_		| The index in _object_ of the first character 
			  that could not be decoded.
| _end_			| One past the end character of this decoding
			  problem.
| _reason_		| The human-readable error text.

The substring _object[start:end]_ is the character or character sequence
that had decoding problems (perhaps I should say 'byte', but this is not
yet Python 3K). You get a character sequence instead of a character in
situations where the first character is a valid start of a multi-byte
sequence, but subsequent characters have an error.
For example, in a theoretical [[UTF-8
http://en.wikipedia.org/wiki/UTF-8]] encoded string, the first two
bytes of the three-byte sequence 0xe0 0x81 0x58 are valid parts of
a three-byte encoding, but the third byte is not. You would get a
UnicodeDecodeError object where _start_ pointed to 0xe0, the first byte,
and _end_ was just past 0x58, the third.

Also worth noting is that when things are being decoded to Unicode, the
first element of the tuple your error handler returns (assuming that you
want to keep going) has to be a *unicode* string.

Given all of this, we can put together a really simple decoding error
handler that just replaces undecodable bytes with backslashed hex
versions of themselves:

>  def bsreplacer(uerr):
>      c = uerr.object[uerr.start]
>      return (u"\\x%x" % ord(c), uerr.start+1)
>  
>  import codecs
>  codecs.register_error("bsreplace", bsreplacer)

Note that we are playing fast and loose with multi-byte sequences;
instead of handling the entire sequence, we just replace the first byte
and restart decoding after it. In some situations this can malfunction
and produce garbled output.

(One would think that the standard "backslashreplace" error handler
would already do this, but unfortunately it doesn't handle decoding
errors, only encoding errors.)