Wandering Thoughts archives

What I'd have liked to hear about Python 3 from the developers

I mentioned in my earlier entry that a different way of framing the Python 3 announcement would have made a big difference in my reactions to Python 3. For peculiar reasons, I think it's worth being explicit about what that would be.

Put simply, my major bad reaction to Python 3 stems from people apparently being expected to port their existing code to Python 3 in order to keep it operational. So for me, the way to make a better announcement about Python 3 is to make it explicit that the developers don't expect that to happen and they understand the implications of that.

So the core message I would have liked to hear goes something like 'please use Python 3 for new code, do consider migrating from Python 2 to Python 3 for code that you're actively developing if it's easy or if the new stuff in Python 3 looks attractive, and otherwise don't worry about your existing code because Python 2 isn't going anywhere; it's a mature and stable release that will be around for a long time'. This is an appealing message for anyone with existing Python 2 code that they have no interest in fiddling with and I don't think it's very far away from the intentions of the Python developers.

The devil's advocate position is to question whether this could have been an honest message (if it had been put forward). There are two parts of this. First, this message is not really attractive if Python 2 still needs to evolve (because the Python developers have made it clear that that's not happening). I happen to think that Python 2 doesn't need anything further, but people may disagree. Second is the issue of major packages that people use. Python 2.7 itself will receive any needed security updates for some years to come, but no one knows how long popular Python packages will still get Python 2 updates. And the Python developers can't make any promises because these packages are developed by third parties.

(This is another area where the lack of easy adaptability (not even compatibility) between Python 2 and Python 3 really hurts people.)

Does Python 2 need to evolve?

Here's a question, somewhat motivated by my entry about my anger at Python 3 (in a way that I'll have to explain later): does Python 2.7.x particularly need to evolve, either in the language or in its standard library?

(By 'evolve' here I do not mean 'turn into Python 3'; I mean 'keep being developed in a way that maintains Python 2 compatibility'.)

There are certainly a number of things that would be nice to have in Python 2 (especially for compatibility with Python 3), but I consider that different from there being any clear lacks or problem points. My relatively off the cuff reaction as a user of Python 2 is that I don't think the core language actively needs anything; I don't see any particular painful issues (at least none that have clear solutions that are backwards compatible with current Python).

The standard library is a little bit more complex. I'm not sure that Python 2 includes all of the batteries you need to deal with a modern web environment, but I generally haven't felt that the standard library was clearly lacking something. In fact I have no real sense of what the Python developers feel is worthy of being in the standard library and what stays outside it. So to the extent that I have an opinion, I'd say that the Python 2 standard library doesn't clearly need anything. I wouldn't be surprised to be totally wrong about this, though.

(Again, there's any number of modules that would be nice to have in the standard library (depending on who you ask). I'm just not sure that any are clearly necessary.)

So on the whole, my answer to my question is that Python 2.7 doesn't particularly need to evolve. It would be nice if it kept moving, but Python 2.7 doesn't have any particular lacks; as it is, it's a perfectly good base for development for years to come.

(There is a widespread view that software is like sharks, either moving or dead. I reject this view in general and think it's perfectly possible for software to either come to a stopping point or to reach a point where it only changes in response to outside changes, like needing updating for updated Internet protocols or whatever. If you are a 'moving or dead' person, then the answer to my question is that Python 2 absolutely has to keep evolving.)

A confession: Python 3 has always made me kind of angry

I have a confession. Although I've written about how various aspects of Python 3 irritate me, I've never come out to admit something: Python 3 makes me kind of angry. Well, okay, let me hastily clarify that. What makes me angry is not so much Python 3 by itself (although various of its defects irritate me) but the apparent views of the Python developers about the relationship between Python 2 and Python 3 that were put forward when Python 3 was released.

(Now, there are a lot of levels of indirection here; my anger is the result of my perceptions of Nth-hand reports of attitudes and remarks. But ultimately this is about my reaction and why, so please add 'as I perceived it' all through this.)

The quick way to summarize this is what I heard when Python 3 came out is 'Python 2.7 is the end of the 2.x line, so get off now'. I very strongly perceived that the Python developers expected everyone to migrate their code to Python 3. As I've written about before migrating my existing code from Python 2 to Python 3 is a lot of work for no gain, yet the Python developers seemed to expect me to not just do this but to be happy about it. So the developers wanted me to do a lot of work just because they'd decided that they wanted to clean up Python in relatively trivial ways.

This made me resentful and angry directly and it also made me angry because it struck me as the height of language designer arrogance, where the designers prioritize what they see as elegance over backwards compatibility and other things that matter to programmers in practice.

(This attitude is not unique to language designers; it is found in any developer who breaks compatibility because they have some sort of better way to do things.)

Worse, not only was I and my existing code was being thrown under a bus by a bunch of smug developers but they were also more or less telling me that it was my own fault that the migration was so hard because most of my code didn't have tests. At the time, I'm pretty sure that I read things to the effect that 'code with good tests should have little problem migrating to Python 3', with the implications about code without good tests rather implied. To put it one unkind way, the attitude that good code and good programmers will have no problem with something hard is a smug Lisp weenie attitude, and if I liked that attitude I might use Lisp. I don't code in Python to be smug and pure, I code in Python to get things done, and to an extent I perceived Python 3 as being a hijacking of a pragmatic language to be smug and pure, to do things the 'right' way no matter the practical cost.

I accept that my anger is partly irrational. I don't think that the Python developers expected or wanted me to take Python 3 and their actions the way that I did, and to some extent my reaction is even unfair. Without actually changing much about the objective situation, I think that a different way of framing the Python 3 announcements would have made me a lot happier with Python 3; if I take the right attitude (and I think it's a realistic attitude), Python 3 is really no big deal. But unfortunately my anger exists and colours my entire interactions with Python 3. So it's high time for me to admit this so that, perhaps, I can move past it and come to appreciate and enjoy Python 3.

One good use for default function arguments

When I wrote about a danger of default function arguments I mentioned that there are cases where they make sense and are useful. Today I'm going to present what I feel is one of them.

To put it simply, one use for default arguments is when you effectively have a bunch of slightly different APIs but it would be awkward to have different functions for them. Not uncommonly you might have too many function variants, the code would be too entwined, or both. Reasonably chosen default arguments effectively give you multiple APIs; one with all the defaults, another with one set of defaults overridden, and so on. You can in fact discover how many different APIs you actually need more or less on the fly, as you write code that uses different combinations of default and non-default arguments.

All of that sounds really abstract, so I'll use an actual example from DWiki. DWiki has a concept of 'views', which are both different ways to present the same underlying bit of the filesystem and different ways of processing URLs. Views have names and handler functions, and there is a registration function for them:

def register(name, factory, canGET = True, canPOST = False,
             onDir = False, onFile = True, pubDir = False,
             getParams = [], postParams = []):
  [....]

This is effectively several APIs in one. Fully expanded, I think it'd be one API that's used to register forms (that's what canPOST, a False value for canGET, getParams, and postParams are for), one API for views of directories only, one API for views of files only, and one API for views that work on both files and directories. As separate functions, each would have a subset of the full arguments for register(). But equally, as separate functions they would all do the same thing and they'd have basically the same name (there is no natural strong name difference between 'register a form view' and 'register a directory view' and so on).

I dislike small variations of things (I'm driven to generalize), so when I was writing DWiki I didn't make separate functions for each API; instead I slammed them together into one function with a bunch of default arguments. The simplest case (with all arguments as defaults) corresponds to what I thought at the time was the most common case, or at least the base case.

(This view of default arguments creating multiple APIs comes from a bit in this talk on Go; reading it crystallized several things in my head.)