Wandering Thoughts archives

2018-02-28

Using Python 3 for example code here on Wandering Thoughts

When I write about Python here, I often wind up having some example Python code, such as the subCls example in my recent entry about subclassing a __slots__ class. Mostly this Python code has been Python 2 by default, with Python 3 as the exception. When I started writing, Python 3 wasn't even released; then it wasn't really something you wanted to use; and then I was grumpy about it so I deliberately continued to use Python 2 for examples here, just as I continued to write programs in it (for good reasons). Sometimes I explicitly mentioned that my examples were in Python 2, but sometimes not, and that too was a bit of my grumpiness in action.

(There was also the small fact that I'm far more familiar with Python 2 than Python 3, so writing Python 2 code is what happens if I don't actively think about it.)

However, things change. Over the past few years I've basically made my peace with Python 3 and these days I'm trying to write new code in Python 3. Although writing my example code here in Python 2 is close to being a reflex, it's one that I want to consciously break. Going forward from now, I'm going to write sample code in Python 3 by default and only use Python 2 if there is some special reason for it (and then mention explicitly that the example is Python 2 instead of 3). This is a small gesture, but I figure it's about time, and it's also probably what more and more readers are just going to expect.

(It looks like I've been doing this inconsistently for a while, or at least testing some of my examples in Python 3 too, eg, and also increasingly linking to the Python 3 version of Python documentation instead of the Python 2 version.)

Actually doing this is going to take me some work and attention. Since I write Python 2 code by reflex, I'm going to have to double-check my examples to make sure that they're valid Python 3 (and that they behave the same way in Python 3). Some of the time this will mean actually testing even small fragments instead of relying on my Python (2) knowledge to write from memory. Also, when I'm checking Python's behavior for something (or prototyping some code), I'll have to remember to run python3 instead of just python or I'll accidentally wind up testing the wrong Python.

(When I wrote my recent entry I was quietly careful to make the example code Python 3 code by including a super() and then using the no-argument version, which is Python 3 only.)

(I'm writing this entry partly to put a marker in the ground for myself, so that I won't be tempted to let a Python 2 example slide just because I'm feeling lazy and I don't want to work out and verify the Python 3 version.)

Python3ForExamples written at 02:24:08; Add Comment

2018-02-25

What Python does when you subclass a __slots__ class is the right answer

Recently on Twitter @simonw sort of asked this question:

Trying to figure out if it's possible to create an immutable Python class who's subclasses inherit its immutability - using __slots__ on the parent class prevents attrs being set, but any child class still gets a mutable __dict__

To rephrase this, if you subclass a class that uses __slots__, by default you can freely set arbitrary attributes on instances of your subclass. Python's behavior here surprises people every so often (me included); it seems to strike a fair number of people as intuitively obvious that __slots__ should be sticky, so that if you subclass a __slots__ class, you yourself would also be a __slots__ class. In light of this, we can ask whether Python has made the right decision here or if this is basically a bug.

My answer is that this is a feature and Python has made the right choice. Let's consider the problems if things worked the other way around and __slots__ was sticky to subclasses. The obvious problem that would come up is that any subclass would have to remember to declare any new instance attributes it used in a __slots__ of its own. In other words, if you wrote this, it would fail:

class subCls(slotsCls):
   def __init__(self, a, b, c):
      super().__init__(a)
      self.b = b
      self.c = c
   [...]

In practice, I don't think this is a big issue by itself. Almost all Python code sets all instance variables in __init__, which means that you'd find out about the problem the moment you created an instance of the subclass, for example in your tests. Even if you only create some instance variables in __init__ and defer others to later, a single new variable would be enough to trigger the usual failure. This means you're extremely likely to trip over this right away, either in tests or in real code; you'll almost never have mysterious deferred failures.

However, it points toward the real problem, which is that classes couldn't switch to using __slots__ without breaking subclasses. Effectively, that you weren't using __slots__ would become part of your class API. With Python as it is today, using or not using __slots__ is an implementation decision that's local to your class; you can switch back and forth without affecting anyone else (unless outside people try to set their own attributes on your instances, but that's not a good idea anyway). If the __slots__ nature was inherited and you switched to using __slots__ for your own reasons, all subclasses would break just like my subCls example above, including completely independent people outside your codebase who are monkey subclassing you in order to change some of your behavior.

(You could sort of work around this with the right magic, but then you'd lose some of the memory use benefits of switching to __slots__.)

Given the long term impact of making __slots__ sticky, I think that Python made the right decision to not do so. A Python where __slots__ was sticky would be a more annoying one with more breakage as people evolved classes (and also people feeling more constrained with how they could evolve their classes).

(There would also be technical issues with a sticky __slots__ with CPython today, but those could probably be worked around.)

SlotsSubclassSurpriseRight written at 01:34:35; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.