Some of my assumptions about Python object allocation

February 14, 2009

In a reddit comment thread I was accurately dinged for being a bit casual in how I talked about object allocation at the end of my previous entry. As the comment notes, calling '.setdefault(k, [])' creates and throws away a new object each time the key already exists, just as '.setdefaults(k, SomeClass())' would.

Well, sort of. One of the things I assume about Python is that allocating and freeing new empty instances of built-in types (such as '[]') is highly optimized and in practice is very cheap, while allocating and freeing instances of Python-level classes is relatively expensive both in memory and in time. I am thus much more casual about churning simple primitive instances than I am about churning user written classes.

(Things like zero-length strings and empty tuples can be optimized even further; since they are unchangeable once created, you can have a single null instance of each and just hand out references to it.)

I think of instances of Python-level classes as expensive to create and destroy for two reasons. First, even a very minimal class instance has a not insignificant amount of overhead; unless it uses __slots__, it has at least a dict instance for its __dict__ and a C-level class instance structure of some sort. Second, if the class has an __init__ method, it must be called, with the overhead in time and object creation and churn that that implies. (And of coure the __init__ can wind up attaching more data on the new instance, making it more expensive.)

Written on 14 February 2009.
« The accumulator mini-pattern and .setdefault()
How CPython optimizes allocations for some built-in types »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Feb 14 00:41:13 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.