Wandering Thoughts archives

A limitation of Python types from C extension modules

It's recently struck me that there is an important difference between types (and classes) created in a Python module and types/classes that come from a C-level extension module.

Suppose that duck typing is not enough and so you really want to make a class that inherits from an outside class (one in another module), yet overrides all of its behavior. This lets you create objects that work the way you need them to but will pass isinstance() checks that are insisting on instances of the original class. Specifically, you want to be able to create instances of your new class without going through the normal object initialization process of your parent class.

(Yes, you'll need to do your own initialization instead to make your version of the behavior all work out, since once you're not using the parent type's initialization you can't assume that any of the parent's other methods keep working.)

If the outside module is a Python module, you can always (or perhaps almost always) do this. If the outside module is a C extension module, there is no guarantee that you will be able to do this (and sometimes you may not even be able to create your descendant class, much less initialize new instances of it). Fundamentally, the reason for this is the same reason as the reason you can't use object.__new__ on everything; the C module is the only thing that knows how to set up the C-level structures for its own objects, so it has to be involved in creating new instances.

This means that types created in C modules can be effectively sealed against descent and impersonation; they simply can't be substituted for in a way that will fool isinstance(). The corollary is that using isinstance() can in some situations be a much stronger guard than you might be expecting.

(It's possible to make a C-level type inheritable; all of the core Python types are C-level types, after all, and you can do things like inherit from list and str and so on.)