Wandering Thoughts archives

How I work on Python 2 and Python 3 with the Python Language Server (in GNU Emacs)

Python is one of the programming languages that I usually edit in GNU Emacs. These days, that means using the Language Server protocol through lsp-mode and the pyls Python language server. Back when I first set this up and wrote early notes on using LSP-based editing of Python, I had not solved the problem of wanting to edit both Python 3 and Python 2 based code in my GNU Emacs sessions.

If you want to do this, it turns out to be important to run either the Python 2 pyls or the Python 3 pyls, depending on whether the file you're editing is written in Python 2 or Python 3. This creates several problems that I had to solve, and eventually did with brute force (if I used Python virtual environments, it probably would be easier).

First, it's obviously necessary to install both versions at once. I install pyls into $HOME/.local/ by using pip's '--user' switch, so I created a .local/bin/py2-pyls subdirectory and manually moved the Python 2 version of pyls from .local/bin into it. This requires me to always update the Python 2 version of pyls before the Python 3 version, which is a bit annoying, but that's life in a world of both versions.

To pick the right version of pyls to run, I use a cover script; the cover script uses various heuristic checks to try to figure out if it's being run in a directory with Python 2 or Python 3 code (it doesn't work for the case of mixed code and I'm not sure that would work in general anyway). I don't have $HOME/.local/bin on my $PATH and GNU Emacs will conveniently just try to run 'pyls' to start the Python LSP server, so I put the cover script in my $HOME/bin.

The most important checks the cover script uses are to look for which version of Python seems to be being run by '#!' lines in any *.py files in the current directory, and whether there are any obvious 'print' statements (which indicate Python 2). If you're going to do this, note that some Python programs are installed with their '#!' line being '#!/usr/bin/env python3' or the like, instead of directly running the Python interpreter. I missed this in the first version of the cover script because all of our Python scripts directly use '#!/usr/bin/python3'.

(Also, the default Python version for my cover script is Python 3, because Python 3 is what I'm writing all my new Python code in.)

All of this is basically a hack but it works pretty well for me, especially in combination with how I'm dealing with my Python indentation problem. The result is a pretty seamless LSP-based Python editing experience in GNU Emacs where everything basically works. I'm not sure I'm sold on the whole LSP-based experience for editing Python, but that's not the fault of my hacks.

Switching to the new in-kernel WireGuard module was easy (on Fedora 31)

One of the quietly exciting bits of recent kernel news for me is that WireGuard is now built in to the Linux kernel from kernel 5.6 onward. I've been using a private WireGuard tunnel on my Fedora machines for several years now, but it's been through the additional COPR repository with an additional DKMS based kernel module package, wireguard-dkms. Among other things, this contributed to my multi-step process fo updating Fedora kernels.

When I first updated to a Fedora 5.6 kernel, I wondered if I was going to have to manually use DKMS to remove the DKMS installed WireGuard module in favour of the one from the kernel itself. As it turned out, I didn't have to do anything; current versions of the COPR wireguard-dkms package have a dkms.conf that tells DKMS not to build the module on 5.6+ kernels. Updating to a 5.6 kernel got me a warning from DKMS that the WireGuard DKMS couldn't build on this kernel, but that was actually good news. After a reboot, my WireGuard tunnel was back up just like normal. As far as I can tell there is no difference in operation between the DKMS WireGuard version and the now in-kernel version except that I have one fewer DKMS module to rebuild on kernel updates.

(The one precaution I took with the COPR wireguard-dkms package was to not install any further updates to it once I'd updated to a 5.6 kernel, because that was the easiest way to keep a WireGuard module in my last 5.5 kernel in case I wanted to fall back.)

After I'd gone through enough 5.6.x Fedora kernel updates to be sure that I wasn't going back to a 5.5 kernel that would need a WireGuard DKMS, I removed the WireGuard DKMS package with 'dnf remove wireguard-dkms'. Then I let things sit until today, when I did two more cleanup steps; I disabled the WireGuard COPR repository and switched over to the official Fedora package for WireGuard tools with 'dnf distro-sync wireguard-tools'. Somewhat to my surprise, this actually installed an updated version (going from 1.0.20200102 to 1.0.20200319).

(I believe that dnf hadn't previously recognized this as an upgrade because of a difference in RPM epoch number between the two package sources. This may be deliberate so that COPR packages override regular Fedora packages at all times.)

PS: Now that WireGuard is an official part of the Fedora kernel, I feel that I should do something to set up a WireGuard VPN on my work laptop. Unfortunately this really needs a WireGuard VPN server (or touchdown point) of some sort at work. We don't currently have one and the state of the world makes it unlikely we'll deploy one in the near future, even for private sysadmin use.