Altering a Python function's local variables with a trace function

Some time ago I wrote that there was no way to change a function's local variables from outside it (well, specifically their name bindings). As it turns out, I'm wrong; there is one way to do it by going in the back door, although it's not a useful way.

There is specific code in CPython that allows a trace function to completely alter a function's local variables and even function arguments; this (C) code specifically reloads the internal interpreter version of local variables from the frame.f_locals dictionary when your trace function returns into the CPython interpreter. This has to be done from a trace function, and it has to be done while the function you want to modify is running; you cannot reach up into the function that called your function and modify its variables.

(You may be able to give it a local tracing function by getting its frame object and then assigning a suitable function to frame.f_trace, but I haven't tested this.)

What you can do to local variables and function arguments is relatively unrestricted. You can change values, you can unbind variables (so that access to them will get UnboundLocalError), and you can bind variables that are currently unbound. However, as expected, you cannot add new local variables; attempts to do so are ignored.

Unfortunately, there is a further complication: you have to access frame.f_locals in a very special way in order to have your modifications work. The problem is that every time you access the f_locals element of a frame object, some C code gets called; this C code re-populates the dictionary from the current internal locals and then returns it to you. This means that if you repeatedly access f_locals directly, you discard your previous changes on each access (as the real version of the locals is only updated from the dictionary when your trace function returns).

For example, suppose that you write:

def localtrace(evt, frame, arg):
  frame.f_locals['a'] = 10
  frame.f_locals['b'] = 20
  return None

If you arrange to hook this up, you will discover that only your change to b has taken; your change to a has disappeared. (It gets worse if you then desperately attempt to debug this by inserting a 'print frame.f_locals' statement in your local trace function; all your changes will disappear.)

The way to get around this is to dereference f_locals exactly once, by immediately binding your own local name for it:

def localtrace(evt, frame, arg):
  d = frame.f_locals
  d['a'] = 10
  d['b'] = 20
  return None

This will work, changing both a and b as you expect.

(I suspect that at least some Python debuggers are currently falling victim to this dark corner.)

As a trivia note, the same C code that is used to update the real function locals for tracing functions is also invoked if you do 'from module import ...' in a function, since the import has the same need to update the function's local variables.

As an extra special trivia note, profile functions can also do this since they (currently) go through the same low-level C code as trace functions. But really, you don't want to go there.

Sidebar: How to actually turn off local tracing

According to the fine documentation, returning None from a local tracing function will turn off tracing for the remainder of the function. According to my actual testing, this is not the case (and reading trace_trampoline in Python/sysmodule.c confirms it); while returning a new local trace function works, returning None is just ignored.

Until this bug is fixed, you will need to 'turn off' tracing by returning a do-nothing trace function from your real local tracing function. A suitable one is 'lambda x, y, z: None'.

My firm impression from both of these issues is that tracing functions and their access to function local variables are a very, very dark corner of the CPython interpreter, and you meddle in it at your peril. It seems clear that not very many people have ever tried to do anything with it, and there may be other problems too.