Wandering Thoughts archives

Abusing Python frame and code objects

Suppose that you want to find the value of the first argument to the function that called your caller. While the inspect module documents the types involved, it doesn't really explain what the members are; here's what I've worked out.

Start with a frame, for example from calling inspect.currentframe() to get your own frame and then backing up a couple of frames to your caller's caller via f_back. The values of all of the function's local variables are in f_locals, a name to value dictionary, but that doesn't tell you which ones are the function arguments and which are ordinary local variables. For that you need to go to the function's code object via f_code.

The names of all the function's local variables are in the co_varnames tuple, in the order that they occur; this puts the function arguments at the front. How many function arguments there are is co_argcount; this includes keyword arguments but not *args or **kwargs stuff.

(There is a complicated naming scheme to handle nested arguments that I am not going to get into here. If you need to care, you should use the functions in the inspect module.)

So let's put all that together and write a function the returns true if its caller's caller had a given first argument (ignoring nested arguments and so on):

import inspect

def isfirstarg(obj):
    fr = inspect.currentframe()
    try:
        fr = fr.f_back.f_back
        if fr is None:
            return False
        if not fr.f_locals:
            return False
        fl = fr.f_locals
        fc = fr.f_code
        if fc.co_argcount < 1:
            return False
        fa = fl[fc.co_varnames[0]]
        return bool(fa is obj)
    finally:
        del fr

Can you use this to solve the mixin over-tracing problem? Probably not; there are any number of plausible functions that take an object as their first argument, despite not being methods. If you assume your code follows conventions, you can avoid false positives by making isfirstarg() also check that the name of the first argument is 'self'.

(As before, all of this is probably CPython specific, especially since a bunch of this is intimately tied to the CPython bytecode format.)

Attribute tracing as a mixin class

Attribute tracing as a standalone class has limits if you want to trace access to all objects of a particular class, not just a few of them; you wind up having to hunt down all the places where you manufacture new instances and modify them and so on. So let's try an attribute tracer as a mixin:

def _getname(obj):
  cl = obj.__class__
  return "<%s.%s>" % \
         (cl.__module__, cl.__name__)

def _repon(obj, n):
  t = obj._t_il
  return (not t) or (n in t)

class GTrace(object):
  _t_il = None
  def __getattribute__(self, n):
    if n in ("__class__", "_t_il"):
      return super(GTrace, self).__getattribute__(n)
    if _repon(self, n):
      print "get %s.%s" % (_getname(self), n)
    return super(GTrace, self).__getattribute__(n)

class Trace(GTrace):
  def __setattr__(self, n, v):
    if _repon(self, n):
      print "set %s.%s" % (_getname(self), n)
    super(Trace, self).__setattr__(n, v)
  def __delattr__(self, n):
    if _repon(self, n):
      print "del %s.%s" % (_getname(self), n)
    super(Trace, self).__delattr__(n)

(You select what attributes are traced by creating a subclass that has _t_il set to something, and then mixing in that instead of GTrace or Trace.)

Unfortunately, it turns out that this simple mixin implementation is not too useful. The problem is that it gives you too much information; because it is mixed in to the class itself, it reports on access to attributes even from code inside the class. The result is that you are probably going to drown in information, unless you're only interested in access to things that are only used from the outside.

Being more selective would require being able to work out if the attribute access is being done by a class method or by an external function. Unfortunately there is no direct access to this information; it would have to be inferred from frame and code information.

Finding the name of your caller in Python

One of the things that would be useful in an access tracer is reporting not just what was accessed, but where it was accessed from. A basic starter for this is knowing what function called you.

CPython has enough introspection faculties to give you this information, but it's not entirely documented how to dig it out. The big tool for this is the inspect module, and we can use it to define a function:

import inspect

def getcallerinfo():
    fr = inspect.currentframe()
    try: 
        fr = fr.f_back.f_back
        if fr is None:
            return "<no caller>"
        fi = inspect.getframeinfo(fr, 0)
        if fi[2] == "<module>":
            return "(%s:%d)" % (fi[0], fi[1])
        else:
            return "%s() (%s:%d)" % (fi[2],
                                     fi[0], fi[1])
    finally:
        del fr

This returns something like 'barfunc() (foo.py:30)', which means that the function that called getcallerinfo() was called by barfunc(), specifically the code at line 30 of foo.py.

Technically we don't need to call inspect.getframeinfo() to obtain this information (we could just use fr.f_code.co_filename and so on directly, since they're documented), but it handles some peculiar magic with the file name for us.

(Needless to say, this is probably specific to CPython; I would be surprised if the same code worked in Jython or IronPython or the like. Since all of this is documented, it is at least likely to be portable across different versions of CPython and to keep working in future ones.)

A simple Python class to trace access to object attributes

Every now and then I run into a situation where I want to know what attributes on an object are getting accessed when. Fortunately it's pretty easy to knock together something to do this, in a variant on the mutating proxy pattern.

Here is a sample implementation, as a Tracer class:

def _repon(obj, n):
  return (not obj._t_il) or \
         (n in obj._t_il)

class Tracer(object):
  def __init__(self, real,
               name = None, monitor = None):
    self._t_ro = real
    if name:
      self._t_nm = name
    else:
      cl = real.__class__
      self._t_nm = "<%s.%s>" % \
                   (cl.__module__, cl.__name__)
    self._t_il = monitor

  def __getattr__(self, n):
    if _repon(self, n):
      print "get %s.%s" % (self._t_nm, n)
    return getattr(self._t_ro, n)

  def __setattr__(self, n, v):
    if n in ("_t_nm", "_t_ro", "_t_il"):
      super(Tracer, self).__setattr__(n, v)
      return
    if _repon(self, n):
      print "set %s.%s" % (self._t_nm, n)
    setattr(self._t_ro, n, v)

  def __delattr__(self, n):
    if _repon(self, n):
      print "del %s.%s" % (self._t_nm, n)
    delattr(self._t_ro, n)

You use it by replacing the original object with Tracer(obj). The optional name argument will be used as the object's name in output; the optional monitor argument is a list of attributes to monitor access to (if omitted, all attributes are monitored).

There's a couple of limitations in this approach:

• 'get' is printed when you do hasattr(), because hasattr() works by getting the attribute and seeing if that throws an error.
• you can't tell a method call from an attribute access, because Python doesn't really distinguish them.

As a side note, the _repon function is not a class function in order to keep it out of the object namespace. I could keep the three variables out too if I wanted to work hard enough, by using a closure, but that would complicate and obscure the code.

(Disclaimer: the indentation has been shrunk and the variable names shortened in the interests of not overflowing WanderingThoughts' margins. I do not write real code that is this condensed.)