Wandering Thoughts archives

Fake versus real metaclasses and what a fully functional metaclass is

Lately I've become a little bit obsessed with the question of whether you can create a fully functional metaclass that doesn't inherit from type (partly this was sparked by an @eevee tweet, although it's an issue I brushed against a while back). It's not so much that I want to do this or think that it's sensible as that I can't prove what the answer is either way and that bugs me. But before I try to tackle the big issues I want to talk about what I mean by 'fully functional metaclass'.

Let's start with some very simple metaclasses, one of which inherits from type and one of which doesn't:

class M1(type):
   def __new__(self, name, bases, dct):
      print "M1", name
      return super(M1, self).__new__(self, name, bases, dct)

class M2(object):
   def __new__(self, name, bases, dct):
      print "M2", name
      return type(name, bases, dct)

class C1(object):
   __metaclass__ = M1

class C2(object):
   __metaclass__ = M2

M2 certainly looks like a metaclass despite not inheriting from type (eg if you try this out you can see that it is triggered on the creation of C2). But appearances are deceiving. M2 is not a fully functional metaclass (and there are ways to demonstrate this). So let me show you what's really going on:

>>> type(C1)
<class 'M1'>
>>> type(C2)
<type 'type'>

(We can get the same information by looking at each class's __class__ attribute.)

The type of a class with a metaclass is the metaclass while the type of a class without a metaclass is type, and as we can see from this, C2 doesn't actually have a metaclass. The reason for this is that M2 created the actual class object for C2 by calling type() directly, which does not give the newly created class a metaclass (instead it becomes a direct instance of type). If all you're interested in is changing a class as it's being created this may not matter, or at least you may not notice any side effects if you don't subclass your equivalent of C2. In this example M1 is what I call a fully functional metaclass and M2 is not. It looks like one and partly acts like one, but that is an illusion; at best it can do only one of the many things metaclasses can do. A fully functional metaclass like M1 can do all of them.

Now let's come back to a demonstration that M2 is not a real metaclass. The most alarming way to demonstrate this is to subclass both classes:

class C3(C1):
   pass
class C4(C2):
   pass

If you try this out you'll see that M1 is triggered when C3 is created but M2 is not triggered when C4 is created.

This is very confusing because C4 (and C2 for that matter) has a visible __metaclass__ attribute. It's just not meaningful after the creation of C2, contrary to what some documentation sometimes says. Note that this is sort of documented if you read Customizing class creation very carefully; see the section on precedence rules, which only talks about looking at a __metaclass__ attribute in the actual class dictionary, not the class dictionaries of any base classes.

Note that this means that general callables cannot be true metaclasses. To create a true metaclass, one that will be inherited by subclasses, you must arrange for the created classes to be instances of you, and only classes can have instances. If you have a __metaclass__ of, say, a function, it will be called only when classes explicitly list it as their metaclass; it will not be called for subclasses. This is going to surprise everyone except experts in Python arcana, so don't do that even if you think you have a use for it.

(If you do want to customize only classes that explicitly specify a __metaclass__ attribute, do this check in your __new__ function by looking at the passed in dictionary. Then people who read the code of your metaclass have a chance of working out what's going on.)

I will admit that Python 3 cleaned this up by removing the magic __metaclass__ attribute. Now you can't be misled quite as much by the presence of C2.__metaclass__ and the visibility of C4.__metaclass__. To determine whether something really has a metaclass in Python 3 you have no choice but to look at type(), which is always honest.