Wandering Thoughts archives

2015-02-18

I wish Python didn't allow any callable to be a 'metaclass'

One of the things that any number of discussions of metaclasses will tell you is that any callable object can be a metaclass, not just a class. This is literally and technically true, in that you can set any callable object (such as a function) as a class's metaclass and it will be invoked on class creation so that it can modify the class being created (which is one of the classical uses of metaclasses).

The problem is that modifying classes as they are being created is the least of what metaclasses can do. Using a callable as your 'metaclass' means that you get none of those other things (all of which require being a real metaclass class). And even your ability to modify classes as they're being created is incomplete; using a callable as your metaclass means that any subclasses of your class will not get your special metaclass processing. This may surprise you, the users of your code and your 'metaclass', or both at once. Unfortunately it's easy to miss this if you don't know metaclasses well and you don't subclass your classes (either in testing or in real code).

I understand why Python has allowed any callable to be specified as the metaclass of a class; it's plain convenient. In the simple case it gives you a minimal way of processing a class (or several classes) as they're being created; you can just write a function that fiddles around with things and be done with it; you don't need the indirection and extra code of a class that inherits from type() and has a __new__ function and all of that. It also at least looks more general than restricting metaclasses to be actual classes.

The problem is that this convenience is a trap lying in wait for the unwary. It works only in one place and one way and doesn't in others, failing in non-obvious ways. And if you need to convert your callable into a real metaclass because now you need some additional features of a real metaclass class, suddenly the behavior of subclasses of the original class may change.

So on the whole I wish Python had not done this. I feel it's one of the rare places where Python has prioritized surface convenience and generality a little too much. Unfortunately we're stuck with this decision, since setting metaclass to any callable is fully documented in Python 3 and probably can't ever be deprecated.

PS: Note that Python is actually inconsistent here between real metaclass classes and other callables, since a metaclass that is a class will have its __new__ invoked, not its __call__, even if it has the latter and thus is callable in general. This is absolutely necessary to get metaclass classes working right, but that this inconsistency exists is another sign that this whole 'any callable' thing shouldn't be there in the first place.

Sidebar: the arguments to your metaclass callable

The arguments for a general callable are slightly difference from the arguments a real metaclass __new__ receives. You get called as:

def metacls(cname, bases, cdict):
    return type(cname, bases, cdict)

If you want to call type.__new__ directly, you must provide a valid metaclass as the first argument. type itself will do, of course. Using a metacls() function that shims in an actual class as the real metaclass is beautifully twisted but is going to confuse everyone. Especially if your real metaclass has a __new__.

(If your real metaclass has a __new__, this will get called for any subclasses of what you set the metaclass function on. I suppose you could abuse this to more or less block subclassing a class if you wanted to. Note that this turns out to not be a complete block, at least in Python 3, but that's another entry.)

python/MetaclassCallableIssues written at 01:32:34; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.