Wandering Thoughts archives


A problem in Python's implementation of closures

Python's implementation of closures for inner functions has a well known problem: you can't mutate the binding of captured outer variables. In other words, the following code does not work:

def cfunc(a):
  def _if(b):
    a = a + b
    return a
  return _if

There is nothing in the semantics of Python that require this result. Unlike the case of writing to a global variable in a function, what the a variable refers to in the scope of _if is completely unambiguous at all times.

I could make excuses for CPython, but the problem is pretty much there deliberately; while there is a special bytecode instruction (LOAD_DEREF) to read the value of captured variables in a closure, there is no bytecode instruction to write to them. In the absence of the ability to do anything else, the interpreter does its standard thing and treats any variable that is stored to in the function as function local (barring a global declaration).

The careful phrasing I have had to use in the first paragraph shows the way around this problem: while you can't change the binding of captured outer variables in a closure, you can mutate their value directly if the type of their value allows this. The classical way to do this is to make the desired variables into arrays, and then mutate the array contents. So we would write cfunc as:

def cfunc(a):
  t = [a]
  def _if(b):
    t[0] = t[0] + b
    return t[0]
  return _if

This version does what we want it to, at the expense of a certain amount of ugliness.

(Credit where credit is due department: I think I first saw the array trick in the sample WSGI server code in its specification.)

python/ClosureProblem written at 23:24:51; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.