Wandering Thoughts archives

The (future) problem with Python 2.7

To put it one way, the future problem with Python 2.7 is that the Python developers have very emphatically said that there will not be a Python 2.8. That's not a problem by itself, but it becomes a problem when combined with what I noted in Python 3 from a sysadmin's perspective, which is that Python 2 is all but certain to stay in Linux distributions for at least five years or so (and quite possibly longer). And not just stay there as an option; for a significant amount of that time, Python 2 will be the primary version of Python. It will be the version of Python that various system tools are written in, it will be the version of Python with all of the extension modules and packages, and it will very likely be the dominant version for actual user code.

(This follows from what I noted in that entry, which is that Linux distributions are only just now starting to think about moving some of their own tools from Python 2 to Python 3. And this is for fast-moving Linux distributions on the leading edge, not the slower moving Linux distributions with long term support.)

Five years (or more, since Python 2.7.2 is already almost a year old) is a long time for a language and a standard library to go without bugfixes, changes and improvements. It's also quite an opportunity for the gap between Python 2.7 and Python 3.x to widen, since even things from Python 3 that could be brought back to the Python 2.x series no longer will be (a bunch showed up in Python 2.7.2, but they're the last).

From some viewpoints, this is a feature; Python 2 is for legacy code, things that people are not interested in migrating for whatever reason, and anyone who wants new features should be using Python 3. From the other side, this is a naive position; the dominant and in many ways pragmatically better version of Python is still Python 2 (because, among other things, of far wider support in packages and addon modules), so new code will continue to be written in it for years. That is, in a language version that has been declared obsolete and at the end of its life.

I don't have any answers here. I just have the gut feeling that this is going to be a problem, that a significant number of people are going to keep programming in Python 2 and will not like the situation.

(Unless things go horribly wrong with the slow migration to Python 3, I don't think that the situation will go so far that Linux distributions will do their own Python 2.7.2+ or 2.7.3 release. I expect people to be vaguely disgruntled, not in active revolt. Still, a bunch of people being vaguely disgruntled doesn't do your programming language any favours.)

Using and understanding Python metaclasses (an index)

Most explanations of Python's metaclasses I've seen start by trying to explain the background and perhaps how metaclasses work, and only then do they start into what you can do with them. This approach doesn't work for me and I kept bouncing off these explanations, so I wrote a series of entries from the other way around; I started with what you can use metaclasses for and only later explained everything else. This entry is an index to this effort.

(I feel one of the reasons metaclasses are such tricky and slippery things is that they can be used to do a number of different jobs that are not obviously related to each other. It turns out there is a reason for all of these things to be clumped together, but to understand it you need quite a lot of background first. So you might as well begin with what you can do with metaclasses and then later explore why they can do all of these things as you get interested.)

To start with is my four part series on what you can do with metaclasses (in order of increasing obscurity):

• changing a class as it's being created, which is the classical case and most of what people use metaclasses for in practice.
• doing something when a class is called to create class instances; many writeups fold this in with the first use, but it's actually kind of different in subtle ways.
• adding and controlling class-only attributes, where we are getting into obscure territory.
• and finally supplying a class's special methods.

For an example of using metaclasses, I've written a namespace metaclass for using (or abusing) classes as namespaces. This uses a metaclass for the first case, changing a new class as it's being created.

Then there's a series of entries on the background of metaclasses, at two levels. At the high level is my explanation of why metaclasses work, which covers the high level features of Python that allows it to have metaclasses. At the low level I have a series of entries on the actual mechanics:

• what (new-style) classes are in Python, with a supplement on understanding isinstance() on classes.
• understanding what types really do, which mentions in passing the important information that a class with a metaclass has that metaclass as its type.
• how Python looks up attributes when you do obj.attr, which is necessary to understand how several parts of metaclasses work.

These entries all have links to earlier entries with even more low level details, delving right down into the guts of how the CPython interpreter implements things like representing classes in memory.

By the end of all of this, you should not only know what you can do with metaclasses (even if you have no use for a lot of the possibilities) but you should also see why all of the different things are all aspects of a general set of mechanisms that runs through much of new-style Python classes.

(Of course, the expected irony of writing an explanation of metaclasses that I could read and understand is that I don't need it any more.)