Wandering Thoughts archives

The challenges of having true constants in Python

In his article What is Actually True and False in Python?, Giedrius Statkevičius takes Python to task for not having genuine constants in the language and making True and False into such constants, instead leaving them as changeable names in the builtins module. This provides a convenient starting point for a discussion of why having true constants in Python is a surprisingly difficult thing.

One of the big divisions between programming languages is what variable names mean. Many languages are what I will call storage languages, where a variable is a label for a piece of storage that you put things in (perhaps literally a chunk of RAM, as in C and Go, or something more abstracted, as in Perl). Python is not a storage language; instead it's what I'll call a binding language, where variables are bindings (references) to anonymous things.

In a storage language, variable assignment is copying data from one storage location to another; when you write 'a = 42' the language will copy the representation of 42 into the storage location for a. In a binding language, variable assignment is establishing a reference; when you write 'a = 42', the language makes a a reference to the 42 object (this can lead to fun errors).

One result of that is that constants are different between the two sorts of languages. In a storage language what it means to make something a simple constant is relatively straightforward; it's a label that doesn't allow you to change the contents of its storage location. In a binding language, a constant must be defined differently; it must be something that doesn't allow you to change its name binding. Once you set 'const a = 42', a will always refer to the 42 object and you can't rebind it.

In Python, what names are bound to is not a property of the name, it is instead a property of the namespace they are in (which is part of why del needs to be a builtin). This means that in order for Python to have true constants, the various namespaces in Python would need to support names that cannot be re-bound once created with some initial value. This is certainly possible, but it's not a single change because there are at least three different ways of storing variables in Python (in actual dicts, local variables in functions, and __slots__ instance variables) and obviously all of them would need this.

You also need some way to support reloading modules, because this normally just runs all of the new module's code in the existing namespace. People will be unhappy if they can't change the value of a module level constant by reloading the module with a new version, or even convert a constant into an ordinary variable (and they'd be unhappier if they can't reload modules with constants at all).

Because the namespace of builtins is special, it would probably not be all that difficult to support true constants purely for it. In theory this would give you constants for True and False, but in practice people can and normally will create module-level versions of those constants with different values. In fact this is a general issue for any builtin constants; if they're supposed to genuinely be constants, you probably don't want to let people shadow them at the module level (at least). This requires more magic for all of the various ways of writing names to module level globals.

One more complication is that Python likes to implement this sort of thing with a general feature, instead of specific and narrowly tailored code. Probably the most obvious general way of supporting constants would be to support properties at the module level, not just in classes (although this doesn't solve the shadowing problem for builtin constants and you'd need an escape for reloading modules). However, there are probably a bunch of semantic side effects and questions if you did this, in addition to likely performance impacts.

(Any general feature for this is going to lead to a bunch of side effects and questions, because that's what general features do; they have far-reaching general effects.)

There's also a philosophical question of whether Python should even have true user-defined constants. Python is generally very much on the side that you can monkey-patch things if you really want to; any protections against doing so are usually at least partially social, in that you can bypass them if you try hard. Genuinely read-only names at the module level seem a violation of that, and there are other mechanisms if all we really care about are a few builtin values like True, False, and None.

(Why Python 2 didn't use such mechanisms to make True and False into 'constants' is another entry.)

Sidebar: Straightforward constants versus full constants

So far I've been pretending that it's sufficient to stop the name binding from changing in order to have a constant (or the storage location for storage languages). As Python people know full well, this is not enough because objects can mutate themselves if you ask them to (after all, this is the difference between a list and a tuple).

Suppose that Python had a magic const statement that made something a constant from that point onward:

alist = [1, 2, 3]
const alist

Clearly this must cause 'alist = 10' to be an error. But does it stop 'alist.pop()', and if so how (especially if we want it to work on arbitrary user-provided objects of random classes)?

One plausible answer is that const should simply fail on objects that can't be dictionary keys, on the grounds that this is as close as Python gets to 'this object is immutable'. People who want to do things like make a dict into a constant are doing something peculiar and can write a custom subclass to arrange all of the necessary details.

(Or they can just make their subclass lie about their suitability as dictionary keys, but then that's on them.)

Link: The Python decorators they won't tell you about

H. Chase Stevens The decorators they won't tell you about (via Hacker News, repeatedly; it's been posted several times) is another view of Python decorators. I'll give you a quote that shows the flavour:

Decorators are often described as "functions which take functions and return functions", a description which is notable in that, technically speaking, not a single word of it is true.

(H. Chase Stevens is right about this, by the way.)

If you're interested in understanding more about what decorators are and some clever (or crazy) things they can be used for beyond the obvious, this is well worth reading. I certainly enjoyed it, even if some of the tricks it shows are things that I'd probably never use in real code.

(Breaking out of conventional views is always useful, in my opinion.)

By the way, the article is a Python Jupyter notebook and as such, you need to turn on JavaScript in order to view it on Github (and apparently people may have problems on mobile devices). An alternate view that doesn't require this is here (via).