Another advantage of Python builtins

July 28, 2008

I've talked before about the speed advantage that Python builtins have. But speed isn't the only way that Python privileges things written at the C level; as dict.setdefault() illustrates, Python makes a useful atomicity guarantee for them that it does not make for methods written in Python itself.

Does this guarantee matter? I think that it does, because it is simultaneously useful and cheap. A concurrent program can avoid locking when dealing with shared data built carefully from builtin types, and duplicating the effects of this in your own Python code would be fairly expensive, especially in a non-threaded program.

(Given the limitations of Python concurrency and thus that most Python programs aren't threaded, performing very well in a non-threaded environment is quite important in practice.)

This also illustrates once again that it can matter a lot to know how things are implemented. If you are writing a threaded program, knowing whether method calls on a shared data structure are concurrency safe or need to be guarded with locks is vital. If a module's documentation gives you no information on thread-safety (and few do), you really need to know how it is implemented, and a straightforward Python implementation of it is not at all equivalent to a C implementation.

(There are some tricky cases where a module is effectively implemented partly in C and partly in Python. Fortunately most such commingled modules seem to do relatively little in Python.)

Written on 28 July 2008.
« The yum versionlock problem
What you can (probably) count on for concurrency in Python »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jul 28 00:27:30 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.