Thinking about a closure confusion

December 26, 2005

Consider the following Python code, which is a very simplified version of certain sorts of real code that people write:

ll = []
for i in range(1,10):
  ll.append(lambda z: i+z)

print ll[0](0), ll[8](0)

A lot of people writing Python code like this for the first time expect to see it print '1 9', when in fact it prints '9 9'.

What I think is going on here is that people are thinking of closures as doing 'value capture' instead of what I will call 'slot capture'. In value capture the closure would capture i's current value, and things would work right. In 'slot capture' the closure captures i's slot in the stack frame and uses it to fish out the actual value when the closure runs. Since i always uses the same slot on every go through the for loop, every lambda captures the same slot value and thus afterwards will evaluate to the same thing.

Slot capture is harder to think about because you have to know more about language implementation; in this case, you need to know what does and doesn't create a new, unique stack frame. For example, this slightly more verbose version of the code does work right:

def make(i):
  return lambda z: i+z

ll = []
for i in range(1,10):

Here the make function is needed for nothing more than magically forcing the creation of a new unique stack frame, with the net effect of capturing the value of each i in the lambdas. Is it any wonder that people scratch their heads and get this wrong every so often?

You can think about this not as stack frames but as scopes. This may make the make() example clearer: functions have a different scope from their callers, but the inside of a loop is in the same scope as outside it. (There are some languages where this is not true, so you can define variables inside a loop that aren't visible after it finishes. Even then, you may or may not get a new stack frame every time through the loop. Aren't closures fun?)

This sort of closure confusion is not restricted to Python; here is an example of the same issue coming up in Javascript, in a real version of my Python example.

Scope rules can get quite interesting and complicated, and of course they interact with closures in fun ways. For example, Javascript Closures has a long writeup of the Javascript scope rules, which are somewhat more exciting than the Python ones. (It also has nice examples of the (abstract) implementation details.)

Written on 26 December 2005.
« Weekly spam summary on December 24th, 2005
A thought about free software licenses »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Dec 26 00:07:27 2005
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.