A nice illustration of the cost of creating and destroying objects in Python

August 12, 2012

Here are two functions (they may look familiar):

def xiter(max):
  for _ in xrange(max):

from itertools import repeat
def riter(max):
  for _ in repeat(None, max):

The first thing that makes these functions interesting is that xiter() runs about 1.25 times as slowly as riter(), once you test with a large enough max to drown out startup effects. This is interesting because these functions are almost identical, both in generated bytecode and in what happens at runtime. Both xrange() and repeat() create and return iterator objects, which the for loop then traverses. Both iterator objects are implemented in C, as are xrange() and repeat() themselves, and the other operations are the same between both functions.

In short, the difference between these two functions is how the two iterator objects behave. And the big difference here is object allocation and deallocation; repeat()'s iterator returns the same pre-existing object max times while xrange()'s iterator returns max different integer objects, most of which are created and then destroyed. While it's possible that the C code for one is much worse than the C code for the other, it's reasonable to assume that this 1.25 times performance difference between the two functions is entirely due to the extra overhead of allocating and deallocating all of those integer objects. This is about as pure an illustration of the cost of object creation and deletion as you could ask for.

(Note that the performance difference is not due to the overhead of having a lot of integer objects around. Only one integer object is alive at any given time in the xiter() version.)

Written on 12 August 2012.
« The CPython bytecode difference between iteration and looping
When and where I fell out of love with Firefox »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Aug 12 23:47:22 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.