Wandering Thoughts archives

Why you can't use object.__new__ on everything

Here's an interesting error message, somewhat slimmed down:

>>> object.__new__(dict)
TypeError: object.__new__(dict) is not safe, use dict.__new__()

At first blush this seems a peculiar artificial limitation: object is the root of the Python type hierarchy and thus its __new__ is the root version of creating new objects, so why doesn't it work?

Like the last peculiarity this is ultimately due to how CPython is implemented. We can get an idea of the real problem by knowing the exact limitation, which is that the C-level type whose __new__ is being called must be the first C-level type in the inheritance tree.

What is going on is that each C-level type creates its own objects itself, because the core of an object is an opaque type-specific blob of data that can only be set up by code that knows what's supposed to be in it. So the C code for object.__new__ can only create things that are object instances at their heart, which dict instances are not.

(The actual code could make things work by calling the C code for dict.__new__ instead, but that's not what you asked Python to do so it declines to be clever.)

The same issue is behind the 'multiple bases have instance lay-out conflict' error you get if you try to make a Python class that inherits from two C-level types. Because an object can only have one blob of of that type-specific data, you can only inherit from a single C-level type.