The challenges of having true constants in Python

January 7, 2018

In his article What is Actually True and False in Python?, Giedrius Statkevičius takes Python to task for not having genuine constants in the language and making True and False into such constants, instead leaving them as changeable names in the builtins module. This provides a convenient starting point for a discussion of why having true constants in Python is a surprisingly difficult thing.

One of the big divisions between programming languages is what variable names mean. Many languages are what I will call storage languages, where a variable is a label for a piece of storage that you put things in (perhaps literally a chunk of RAM, as in C and Go, or something more abstracted, as in Perl). Python is not a storage language; instead it's what I'll call a binding language, where variables are bindings (references) to anonymous things.

In a storage language, variable assignment is copying data from one storage location to another; when you write 'a = 42' the language will copy the representation of 42 into the storage location for a. In a binding language, variable assignment is establishing a reference; when you write 'a = 42', the language makes a a reference to the 42 object (this can lead to fun errors).

One result of that is that constants are different between the two sorts of languages. In a storage language what it means to make something a simple constant is relatively straightforward; it's a label that doesn't allow you to change the contents of its storage location. In a binding language, a constant must be defined differently; it must be something that doesn't allow you to change its name binding. Once you set 'const a = 42', a will always refer to the 42 object and you can't rebind it.

In Python, what names are bound to is not a property of the name, it is instead a property of the namespace they are in (which is part of why del needs to be a builtin). This means that in order for Python to have true constants, the various namespaces in Python would need to support names that cannot be re-bound once created with some initial value. This is certainly possible, but it's not a single change because there are at least three different ways of storing variables in Python (in actual dicts, local variables in functions, and __slots__ instance variables) and obviously all of them would need this.

You also need some way to support reloading modules, because this normally just runs all of the new module's code in the existing namespace. People will be unhappy if they can't change the value of a module level constant by reloading the module with a new version, or even convert a constant into an ordinary variable (and they'd be unhappier if they can't reload modules with constants at all).

Because the namespace of builtins is special, it would probably not be all that difficult to support true constants purely for it. In theory this would give you constants for True and False, but in practice people can and normally will create module-level versions of those constants with different values. In fact this is a general issue for any builtin constants; if they're supposed to genuinely be constants, you probably don't want to let people shadow them at the module level (at least). This requires more magic for all of the various ways of writing names to module level globals.

One more complication is that Python likes to implement this sort of thing with a general feature, instead of specific and narrowly tailored code. Probably the most obvious general way of supporting constants would be to support properties at the module level, not just in classes (although this doesn't solve the shadowing problem for builtin constants and you'd need an escape for reloading modules). However, there are probably a bunch of semantic side effects and questions if you did this, in addition to likely performance impacts.

(Any general feature for this is going to lead to a bunch of side effects and questions, because that's what general features do; they have far-reaching general effects.)

There's also a philosophical question of whether Python should even have true user-defined constants. Python is generally very much on the side that you can monkey-patch things if you really want to; any protections against doing so are usually at least partially social, in that you can bypass them if you try hard. Genuinely read-only names at the module level seem a violation of that, and there are other mechanisms if all we really care about are a few builtin values like True, False, and None.

(Why Python 2 didn't use such mechanisms to make True and False into 'constants' is another entry.)

Sidebar: Straightforward constants versus full constants

So far I've been pretending that it's sufficient to stop the name binding from changing in order to have a constant (or the storage location for storage languages). As Python people know full well, this is not enough because objects can mutate themselves if you ask them to (after all, this is the difference between a list and a tuple).

Suppose that Python had a magic const statement that made something a constant from that point onward:

alist = [1, 2, 3]
const alist

Clearly this must cause 'alist = 10' to be an error. But does it stop 'alist.pop()', and if so how (especially if we want it to work on arbitrary user-provided objects of random classes)?

One plausible answer is that const should simply fail on objects that can't be dictionary keys, on the grounds that this is as close as Python gets to 'this object is immutable'. People who want to do things like make a dict into a constant are doing something peculiar and can write a custom subclass to arrange all of the necessary details.

(Or they can just make their subclass lie about their suitability as dictionary keys, but then that's on them.)

Comments on this page:

Perl is not easily classified as either a storage or binding language. It’s sort of a hybrid, and which category you give primacy to will depend on the criterion by which you draw the distinction.

It is most definitely a binding language in terms of the meaning of variable names. But for simple value types it uses what amounts to a single-value mutable container, and due to that mutability, assignment necessarily creates copies – so Perl ends up mostly acting like a storage language. (Though it optimises the copying of long strings by using a CoW mechanism – so copying may result in multiple containers but not multiple copies of the value… (Though that is not visible at the Perl level.)) However, its binding nature has always been exposed in a variety of special cases – and then a few years ago, 5.22 added an explicit rebinding operation.

Not easily condensed into this vs. that…

… and it doesn’t affect your argument anyway. So I’d be inclined to just remove the mention of Perl.

By cks at 2018-01-08 11:56:12:

My binding versus storage dividing line is whether my Python mistake is possible with basic container types. If you write:

thing1 = thing2 = <new whatever>

and then do things that mutate thing2, do you also mutate thing1? In Python you do. In Perl, I believe that you don't; Perl copies the array or hash. Maybe I'm wrong, though, as it's been a while since I touched Perl.

If you use a reference, copying the reference and then operating through it mutates the underlying thing. For fuzzy reasons I don't consider references or pointers to turn something into a binding language, although if their use is pervasive and how things are done, maybe I should. But explicit use of references makes things, well, more explicit; in theory you know you're just copying the reference instead of the actual thing.

By cks at 2018-01-09 02:12:15:

I forgot to say why I mentioned Perl at all, which is that I don't want to give the impression that storage languages are all low level languages. Perl makes a handy counterexample if one can say that it's a storage language, and sadly I don't know of any better ones.

(Go is not quite low level, but it's a lot lower than Perl, especially in people's minds).

It’s still a matter of definitions. There isn’t really an obvious single equivalent of that code in Perl.

If you say @thing1 = @thing2 = () then what’s being assigned to @thing2 is not a container. It’s just a list of values (an empty one in this case). And that first assignment evaluates to a list which is then assigned to @thing1.

You can also say $thing2 = $thing2 = [], and then you do assign a reference to a container. Then it behaves exactly as it does in Python.

Basically, from a Perl perspective, every Python variable is a scalar, and containers never exist without an implicit reference. If you evaluate a variable that is bound to an array, you get the array… as a thing… what Perl would think of as a reference. There’s no reference-taking operator because it’s the default, in a sense.

I recently wrote about this.

But outside of assignment, Perl has other constructs that expose binding without requiring references at all; e.g. on the inside of the block in for my $foo ( $bar ) { ... }, $foo and $bar are names for the same thing – and it can well be just a string. Mutating one of them mutates them both.

If you limit the question to “does assignment bind or copy?” then that seems simpler to answer: in Perl, assignment copies. Except, not really; in scalar context, evaluating an array variable doesn’t copy it, it yields the length of the array. And assigning one array variable to another ends up copying it by way of the fact that assignment to an array imposes list context to the RHS expression and evaluating an array variable in list context destructures it.

Err, you get the idea. It’s a really long detour into the weeds if you try to pin Perl down on this count.

It’s definitely not in the same clear-cut simple category as C and Go. Nor, on the other side, Python.

Written on 07 January 2018.
« Link: The Python decorators they won't tell you about
Link: Some fascinating details of cellular data transmission »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Jan 7 18:38:03 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.