Wandering Thoughts archives


Ints, __slots__, and Python 3

Today, one of my visitors came here after doing a Google search for the error message:

nonempty __slots__ not supported for subtype of 'int'

Right away, I can tell you something about this person; they are either using Python 3 or porting code to Python 3.

(I use 'porting' instead of 'migrating' deliberately.)

At one level, the reason you get this error message is simple and I covered it in HowSlotsWorkI; in short, you are trying to subclass a type with a non-zero __itemsize__, one whose C-level storage is already variable sized, and that doesn't work because of how CPython implements slots. I can tell that this is a Python 3 user because only in Python 3 does int have a non-zero __itemsize__.

Or to put it differently, in Python 2 ints are fixed-size objects and in Python 3 they are not. So, you might ask, why is this so? The answer is that Python 3 fully unified ints and longs.

Python 2's long type is what other languages would call a 'bignum', an arbitrary-length integer number. Like everyone else's bignum implementation, Python 2 stores longs in variable sized objects; this is clearly necessary since a bignum can be arbitrarily large and the actual length of bignums varies considerably. Python 2's int is your ordinary good old fashioned fixed-length integer type (either 32 bits or 64 bits depending on what the underlying C int type is on your platform).

In Python 2, ints and longs are thus different types but are partially unified in that you can freely mix them in arithmetic and an int that grows sufficiently large is automatically converted to a long (however, small longs do not automatically convert back into ints). In Python 3, they have been completely unified to the point where there is only one type, int. Since this type has to represent bignums as well as ordinary small integers, its underlying storage must be variable sized; since it has a variable-sized storage, it can't be used with slots.

(In a sense the way to deal with this issue when porting to Python 3 is simple; you just stop using __slots__ on your subclass of int. Everything works the same and your instances just take up a bit more memory.)

By the way, before anyone freaks out over the clear memory waste of representing all small integers with bignums, well:

>>> import sys
>>> sys.getsizeof(10)
>>> sys.getsizeof(10L)

(This is on a 64-bit Linux machine with Python 2.7; a 32-bit machine with 2.6.5 reports 12 bytes and 16 bytes respectively. Using Python 3 on the same 64-bit machine reports a size of 28 bytes.)

(Some people would quibble over this being called 'unification', in that it's more like the Python 2 'long' type was just renamed to be 'int'. In fact this is pretty much exactly what happened at the C API layer.)

python/IntSlotsPython3k written at 00:14:23; Add Comment

Page tools: See As Normal.
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.