A difference between Python 'structs' and C structs

April 19, 2011

I've written before about how I emulate C structs in Python. This is only an emulation, and one of the differences between the two, one that is sometimes important, is that elements in such a Python struct do not have an order.

Where this comes up for me is that every so often I want to use introspection of a Python struct to automatically map between the Python struct and some flat ordering of the fields, such as a file or a network protocol message. With no inherent order to the fields, I wind up having to specify the order explicitly in some way, which always vaguely annoys me.

I consider this subtly different from my case of lists with named fields, although I could use the latter to deal with this problem. To me, lists with named fields are lists first and have named fields later, whereas these are structs with named fields that happen to (sometimes) have a canonical ordering. Possibly I am thinking too hard about this.

Of course, if you have full freedom you can always force a canonical ordering on a given Python struct if you want. Since you have the field names, you can just sort them alphabetically and declare that the canonical ordering. It may not look very nice, but it does answer the question.

(It will also cause you heartburn when you add a new field and you want it to go at the end of the ordering but of course it doesn't sort that way. Pretty soon you start manipulating the result of sorting the field names and that way lies doom and more doom.)

Sidebar: why this lack of ordering exists

My Python emulation of structs just uses a class, well, instances of a class. Instance elements (and class elements) are unordered in most circumstances because all Python does is shove them in the class or instance dictionary, and dictionaries are themselves unordered. I believe that Python classes that use __slots__ do have an actual ordering to their instance elements, but I have never looked very deeply into how that works.

The necessary disclaimer is that this is how CPython works. Other implementations of Python might give elements an actual order and just fake class and instance dictionaries for you if you look at them.

(Python dictionaries are unordered in CPython because they are implemented with hash tables, and hash tables do not have a predictable or stable ordering.)

Comments on this page:

From at 2011-04-19 07:54:31:

It really, really sounds like you want to emulate C structs with classes with __slots__; why don't you?

Classes with __slots__ do have a canonical ordering of fields encoded in __slots__ itself, regardless of how the implementation decides to store that.

-- DanielMartin, who isn't even trying to remember his password anymore

By cks at 2011-04-19 10:44:51:

Using __slots__ is slightly annoying because you have to make up classes on the fly; you can't simply set attributes on an object the way you can with the simple hack.

Written on 19 April 2011.
« What made X Windows so special
Another reason to avoid using __slots__ in your Python classes »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Apr 19 00:22:10 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.