The increasing mess of libreadline versions in Linux distributions

September 22, 2019

GNU Readline is normally supplied and used by programs as a shared library (even if it's possible to statically link it, almost no one does). Some or perhaps many of those programs are not from the distribution; instead they're your local programs or third party programs. Shared libraries have major and minor versions (and also symbol versioning, but let's ignore that for now). The minor version of a shared library can be changed without upsetting programs linked to it, but the major version can't be; different major versions of a shared library are considered to be entirely different things. If your system has libreadline.so.6 and libreadline.so.8, and you're trying to run a program that was linked against libreadline.so.7, you're out of luck.

(Major shared library differences are required by ABI differences even if the API is the same and the program's source code can be immediately rebuilt against a different version of the shared library with no code changes.)

Unfortunately, two things are true. First, the GNU Readline people apparently do things that change the ABI on a regular basis, which causes new versions of the shared library to have new .so major versions (again, on a regular basis). Second, Linux distributions are increasingly providing an incomplete set of past libreadline shared library versions. This came up with Ubuntu 18.04 and libreadline.so.6, and recently I discovered that Fedora 30 has moved from libreadline.so.7 to libreadline.so.8 and not provided a compatibility package for version 7 (although they do versions for readline 6 and readline 5).

(I'm assuming here that the shared library version is changing due to genuine ABI incompatibility, instead of just the GNU Readline people deciding to call their latest release 'readline 8' and everyone following along in the .so versions.)

Just as with Ubuntu, the net effect is that it's impossible to build a local binary that uses GNU Readline that works on both Fedora 29 and Fedora 30, or even that can survive your system being upgraded from Fedora 29 to Fedora 30. If you upgrade your system, you get to immediately rebuild all programs using GNU Readline. I don't think you can even install the Fedora 29 readline 7 RPM on a Fedora 30 system without blowing things up.

It's my strong opinion that this overall situation is asinine. Linux distributions are not KYSTY environments, where the system packages are only for system use; a Linux distribution can reasonably expect people to build software against system shared libraries. Right now, using GNU Readline in such software is pointing a gun at your own foot, and not using GNU Readline is annoying for people who use your software (people like those readline features, and for good reason).

(At the same time, the Linux distributions are not the only people you can blame. The GNU Readline people are presumably unlikely to do bug fixes and security updates for GNU Readline 7, because they've moved on to GNU Readline 8. Linux distributions don't want to have to take on the burden of maintaining a long tail of GNU Readline versions that are no longer supported upstream.)

As a side note, it's very easy to miss that this has happened to some of your binaries if you only run them once in a while. I generally assume that Linux binaries are quite stable and so don't run around testing and rebuilding things after Fedora upgrades; generally I don't even think about the possibility of things like missing shared libraries.

PS: In their current development versions, Debian appears to have both libreadline7 and libreadline8; older versions of GNU Readline seem to be more spotty in general. The current stable Debian has libreadline7.


Comments on this page:

Package own binaries into own rpms - that will keep deps in order and won't allow to break things easily by upgrading to newer soname like in case of readline.

I am not guilty of trying work around major library revision changes by symlinking other versions to the same name. Someone else broke into my house and used my computer, your honour. I swear.

a Linux distribution can reasonably expect people to build software against system shared libraries

This bites me often. I dream of a distro/package-system that makes old versions of libs and programs a first-class option.

I also dream of other things, like proper multi-arch support. Void Linux's xbps does not support multiarch well -- x86-32 on x86-64 is a bit of a hack and packages can't span multiple arches (as is required for things like Wine 64+32).

Mint/Ubuntu (presumably Debian too?) isn't greatly better from a multiarch user-friendliness point of view. Installing wine isn't enough, you also have to work out what 32-bit libgl packages need to be installed, otherwise your games don't work or they crawl. I can never remember their names.

Linux distributions don't want to have to take on the burden of maintaining

Yeah, in my experiences many distro-maintainers are already stressed and in the thick of it.

@Me: that's a lot more work and it's not flexible.

In my home dir I have a folder full of a few dozen different programs and manually-compiled games. Some of these:

  • are constantly changing & being recompiled
  • do not support system-wide installation support (ie split into /usr/share/xxx, etc)

I also think that holistically "join them if you can't beat them" isn't a great option. What if the big package manager finds a way of using this to cause you problems? Eg hold half of your system back in weird or incompatible ways? I suspect that provided old lib packages will cause less issues (& help requests from users) than recommending users package everything.

By codegeek98@gmail.com at 2024-05-06 13:47:35:

If the library's a must-have, its ABI is unstable, and OS vendors are unable or unwilling to support the corresponding rainbow of old versions... how hard is it for application developers to just bundle the matching readline .so with their applications, anyway?

cf. how programs on Windows, as well as major and well-maintained Python libraries such as lxml, will quietly have the needful DLL/SOs tucked away in the app data

Written on 22 September 2019.
« Why chroot is a security feature for (anonymous) FTP
How we implement per-user CPU and memory resource limits on Ubuntu »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Sep 22 23:51:56 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.