Some ways to add versioning to pickled objects
To follow on from yesterday's entry, suppose that one is using (c)Pickle to save data in your Python program, and you want to version your data somehow. I can think of a number of approaches:
- take the 'pickle as stored JSON' approach; serialize your complex
objects to simple objects (dictionaries, lists, etc), add version
numbers, and only pickle the simple objects. Then you can do all of
the usual version mismatch fixups when you de-serialize the reloaded
simple objects back to your complex objects.
- version your class names; instead of having, say, a
Commentclass and pickling several different versions of it, have a
CommentV2class, and so on. (I imagine that you will want a
Commentabstract class to have all of the common behavior.)
- don't explicitly version things. Take advantage of the fact that
pickle doesn't actually initialize objects as such, just stuffs
data into their
__dict__, and write your object methods such that they will deal with any set of data they could get from any version of your objects. (Renaming instance variables may help.)
The easiest approach to this is probably to call a fixup method on newly loaded objects; this method can then canonicalize old data versions into the current world.
- define a custom
__setstate__method that works out what version of the data that it's restoring based on the contents of the dictionary that it's handed. This is essentially the fixup method approach, just automated, and you have to copy the data onto the object yourself.
All of these have drawbacks, and some of them are ugly. If I had to do any of these I would probably take the 'pickle as stored JSON' approach; although it is one of the more annoying choices (since you write a bunch of code), it is the least ugly.
__setstate__ approach has a pleasing minimalism
but involves a little bit too much magic to make me happy.)