What we'll likely do when Linux distributions drop Python 2 entirely

August 22, 2021

Currently, Linux distributions claim that they want to stop including even a minimal Python 2 at some point, although when that will be isn't clear (the latest Debian and Ubuntu in development versions both seem to still include it). Since we have any number of perfectly functional small Python 2 programs used in managing our systems (and no particular time or enthusiasm for porting them all to Python 3), this presents a potential future problem, never mind the disruption for our users (who may also have such programs). Thus, it seems likely that we will roll our own version of Python 2 even after Linux distributions stop doing so.

Our traditional way of dealing with dropped packages is to simply save a copy of the binary Ubuntu package from the last Ubuntu version (or LTS version) that supported it, then install it ourselves by hand; one can do a similar thing with binary RPMs for Fedora. Unfortunately this isn't really sustainable for Python in specific, because Python uses various shared libraries that keep changing their shared library version and which distributions often don't provide compatibility packages for.

(One big offender is libreadline, which is normally used by interactive python2 through the readline module, but there probably are others.)

This leaves either hand-building and manually installing Python 2 through the old fashioned 'make install' approach, or rebuilding the last distribution source packages ourselves on new distribution versions. The 'make install' approach is brute force and wouldn't naturally give us a /usr/bin/python2 symlink, but we could add that by hand, and we could probably create an install (say into /opt) that only required a tarball to be unpacked on each machine. Rebuilding Ubuntu source packages is somewhat more annoying than I'd like, but it's generally feasible unless they specify narrow ranges of build-time dependencies. I took a look at the current Ubuntu source package for Python 2, and while it looks a little tangled I think it will probably rebuild without problems (at least for a while). So most likely we'll start out by rebuilding the source package and only switch to a fully by hand approach if package rebuilding falls apart.

Various standard Python modules written in C make calls to various third party C APIs; I've already mentioned readline, but there are also (for example) the various database access modules. These APIs are generally quite stable, but this isn't the same thing as "frozen", so at some point in the more distant future some standard Python 2 modules might require changes to keep building and working. If the changes are easy, we might patch our Python 2; if they're more difficult, we might have to start dropping standard modules.

For various reasons, we might not want to keep a /usr/bin/python2 symlink at all, at least not over the long term, even if we continue to have Python 2 itself. It's one thing to support Python 2 for our own programs, with their limited needs for standard modules and so on; it's another thing entirely to have to support Python 2 for our general user population. As a result, we might someday want to drop support for the user population (ie, /usr/bin/python2) without dropping our own support. This might push us to switching from rebuilt source packages to a hand install.

PS: Of course another option is migration to PyPy, which will probably always support Python 2 and is certainly feasible for our own Python 2 programs. But our users will almost certainly want us to provide Python 2 for longer than Ubuntu does, so some degree of do it ourselves Python 2 is likely in our future. And while PyPy is a good implementation of Python 2, but it's not really a substitute for a CPython /usr/bin/python2.

Comments on this page:

Why not just port these scripts to Python 3? If they are small, porting them should be really quick.

You might find that the porting is a good opportunity. At work, we used the transition to cleanup things, like using argparse instead of manually looking at sys.argv, adding debug and dry-run flags, and making the code pylint clean (with some warnings turned off generally or specifically as appropriate—no need to be religious about it). After getting in the habit doing the first one or two, the extra work hardly added that much time and we are extremely pleased with the results. The debug and dry-run flags really help in both development and usage: everyone, but especially new/junior people, can feel more confident in using a tool if they know what it is going to do.

If we had instead spent that same time keeping Python 2 alive, we wouldn’t have gotten those improvements.

By cks at 2021-08-24 17:19:24:

These programs are often critical elements of our overall infrastructure (they handle things like our password propagation), and they're all what I call utility code, which means that we normally spend no time working on them and touching them in any way is almost pure overhead. This leaves me pretty biased toward a general solution that scales trivially to all of our programs, rather than having to deal with them one by one and risk instability and new bugs (from things like surprise Unicode issues; I don't think they'd have any, but I can't be completely sure).

It amuses me how people claim to like Python for various reasons, such as convenience, and then bemoan this issue purposefully thrust upon them by the creators. Why not take the opportunity to switch to a real language, such as Common Lisp, which doesn't invalidate standard code on a whim; Common Lisp programs from thirty years ago work without issue. I'd even help with such a transition.

Written on 22 August 2021.
« Setting up Python LSP support in GNU Emacs is reasonably worth it
Notes on deliberately invoking actions controlled by systemd timers »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Aug 22 23:57:10 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.