2021-08-22
What we'll likely do when Linux distributions drop Python 2 entirely
Currently, Linux distributions claim that they want to stop including even a minimal Python 2 at some point, although when that will be isn't clear (the latest Debian and Ubuntu in development versions both seem to still include it). Since we have any number of perfectly functional small Python 2 programs used in managing our systems (and no particular time or enthusiasm for porting them all to Python 3), this presents a potential future problem, never mind the disruption for our users (who may also have such programs). Thus, it seems likely that we will roll our own version of Python 2 even after Linux distributions stop doing so.
Our traditional way of dealing with dropped packages is to simply save a copy of the binary Ubuntu package from the last Ubuntu version (or LTS version) that supported it, then install it ourselves by hand; one can do a similar thing with binary RPMs for Fedora. Unfortunately this isn't really sustainable for Python in specific, because Python uses various shared libraries that keep changing their shared library version and which distributions often don't provide compatibility packages for.
(One big offender is libreadline,
which is normally used by interactive python2 through the readline
module, but there probably are others.)
This leaves either hand-building and manually installing Python 2
through the old fashioned 'make install' approach, or rebuilding
the last distribution source packages ourselves on new distribution
versions. The 'make install' approach is brute force and wouldn't
naturally give us a /usr/bin/python2 symlink, but we could add that
by hand, and we could probably create an install (say into /opt) that
only required a tarball to be unpacked on each machine. Rebuilding
Ubuntu source packages is somewhat more annoying than I'd like, but
it's generally feasible unless they specify narrow ranges of build-time
dependencies. I took a look at the current Ubuntu source package for
Python 2, and while it looks a little tangled I think it will probably
rebuild without problems (at least for a while). So most likely we'll
start out by rebuilding the source package and only switch to a fully
by hand approach if package rebuilding falls apart.
Various standard Python modules written in C make calls to various third
party C APIs; I've already mentioned readline, but there are also (for
example) the various database access modules. These APIs are generally
quite stable, but this isn't the same thing as "frozen", so at some
point in the more distant future some standard Python 2 modules might
require changes to keep building and working. If the changes are easy,
we might patch our Python 2; if they're more difficult, we might have
to start dropping standard modules.
For various reasons, we might not want to keep a /usr/bin/python2
symlink at all, at least not over the long term, even if we continue
to have Python 2 itself. It's one thing to support Python 2 for our
own programs, with their limited needs for standard modules and so on;
it's another thing entirely to have to support Python 2 for our general
user population. As a result, we might someday want to drop support for
the user population (ie, /usr/bin/python2) without dropping our own
support. This might push us to switching from rebuilt source packages to
a hand install.
PS: Of course another option is migration to PyPy, which will probably always support Python 2 and is certainly feasible for our own Python 2 programs. But our users will almost certainly want us to provide Python 2 for longer than Ubuntu does, so some degree of do it ourselves Python 2 is likely in our future. And while PyPy is a good implementation of Python 2, but it's not really a substitute for a CPython /usr/bin/python2.
Setting up Python LSP support in GNU Emacs is reasonably worth it
When I initially set up GNU Emacs LSP support for Python, I wasn't sure if the effort was going to be worth it for my Python programming (which is currently mostly not using type hints). Although I don't edit a lot of Python these days, I've come to believe that using the current Python LSP server is worth the effort to set it up, although it's not clearly a win the way it is for more static languages like Go.
(I'm not sure I'd set up an Emacs LSP environment if I was only editing Python, but if you have an LSP environment already, adding Python to it isn't much work.)
My most common LSP operations while editing Python code are 'go to definition' and 'see references/uses'. The former is available in some form in the basic Emacs Python mode, although I think it's not as fast (and perhaps not as good). The LSP mode can pop up help about a particular function or method under many circumstances, but often I have to wait a while for it to figure things out. Never the less it does a better job of things like finding the right methods than I would have expected.
(For me and my type hint free Python, LSP doesn't seem to do much useful completion, which is normally a big win in other LSP languages.)
GNU Emacs with the LSP mode does a number of helpful passive things, such as flagging "lint" level problems (sometimes too many of them, depending on what you have enabled) and more serious issues like mismatched types of indentation. The basic Python mode probably does some of this, but not all (at least without extra integrations). The LSP mode offers more features and options than this, but I haven't needed to dip into them yet.
(One of the things that writing this entry is showing me is that I didn't pay much attention to what the basic Python mode could do for me. I doubt I ever tried 'go to definition' before I had LSP mode set up.)
More broadly, I think that the Python LSP server and LSP mode is the most likely future of smart Python editing in GNU Emacs, so I might as well get on board now. I suspect that more people will be putting more effort into both the Python LSP server and LSP support in GNU Emacs than are putting effort into making the GNU Emacs Python mode smarter.
If all of this sounds sort of lukewarm, that's a relatively good reflection of my feelings. I think that GNU Emacs LSP support is a clear win for some languages (my example is Go), but doesn't get you as much for Python and probably other dynamic languages like it. Python with type hints might unlock more LSP-based power.
One drawback of using the LSP mode is that I want to be using Emacs under X, not in a terminal. This is doable even in these days of working from home (I can start a SSH session that forwards X), but it adds an extra step and a bit of friction. Editing in text mode is still workable, but it reduces the LSP mode's power.
PS: Possibly there are tweaks to the Python LSP server configuration, my Emacs LSP configuration, or my Python setup that would give me a better Python LSP experience. Since I don't spend much time editing Python code these days, as mentioned, I haven't tried very hard to improve my environment. Possibly this is a mistake.