Wandering Thoughts archives

An interesting way to shoot yourself in the foot and a limitation of super()

Presented in the traditional illustrated form, for Python 2:

class Base(object):
  def foo(self):
    print "base foo"

class A(Base):
  def foo(self):
    print "A foo"
    return super(A, self).foo()

class B(object):
  def foo(self):
    print "B foo"

old_A = A
A = B

old_A().foo()

(If you think that this is a completely silly and crazy example, consider shimming modules for tests where you might need to transparently wrap another module's class.)

If you are an normal innocent Python programmer, this probably looks like it ought to work; after all, you're using super() exactly as the instructions tell you to. If you run it, though, you will get an interesting error:

TypeError: super(type, obj): obj must be an instance or subtype of type

In many languages, the 'super(A, self)' would have been what I'll call 'early binding'; in the context of foo(), the name 'A' would have been immediately bound to the class itself. Renaming the class later (if it's even possible) would make no difference to the line, because the binding had already been established. Python doesn't work that way. The binding of 'A' to a class is only looked up as foo()'s code is executing, and by that time 'A' points to the wrong thing; it points to the completely unrelated class B, and so the call is actually doing 'super(B, self)' (hence the error message). You can see this directly by using the dis module to inspect the bytecode for foo; it contains a LOAD_GLOBAL to look up the value of A, instead of any direct reference to the class.

In fact Python can't work that way. Because of how class definition works combined with how functions are defined, the name A does not exist when the (method) function foo() is being defined and its code is being compiled, so even if Python wanted to do early binding there's nothing to bind to. This late binding is the only choice Python has.

This has an unfortunate consequence; it makes using super() one of the few places in Python 2 where you absolutely have to know what your class is called (well, have some name for it), where by 'your class' I mean the class that the code is in (as opposed to the class of self, which is easy to determine). Of course, normally this is not an issue.

(super() needs to know this because it has to find where you are in the method resolution order, which requires knowing what class the code is in.)

PS: this issue with super() is fixed in Python 3 through methods beyond the scope of this entry (see here if you really want to see the sausage being made).