My views on when you should use the official upstream versions of software

November 23, 2020

Yesterday I wrote about how sometimes it's best to use the upstream versions, with the story of Prometheus here as the example for why you can be pushed into this despite what I've said about the problems inherent in this. But I didn't write anything about when you should do this versus when you should stick with whatever someone else is providing for you (usually your operating system distribution). There's no completely definite answer, partly because everyone's situation is a bit different, but I have accumulated some views here.

In general, what we really care about is not where the software comes from but how well curated what you're getting is, because curating software is work and requires expertise. Usually the best source of curation is packages provided by your OS, which typically add an extra layer of quality assurance over the upstream releases (or over people who put together low-curation OS specific packages from upstream releases). OS packages also come with automatic updating, or at a minimum central notification of updates being available so that you don't have to hunt down odd ways of keeping informed about updates.

The obvious reason to use the upstream version (building it yourself if necessary) is when there's no other option because, for example, you use Ubuntu and there are no official packages of it. Whether you want to do this depends on how much you need the package, how easy it is to build and operate, and how likely it is to have problems. We do this for some of the Prometheus exporters we use, but they have the advantage of being simple to build (Go programs usually make this easy), simple to operate, and extremely unlikely to have problems. They also aren't critical components, so if we had to drop one because of problems it wouldn't be a big deal. We also do this for Grafana, because we absolutely have to have Grafana and there is no Ubuntu package for it, so our best option left is the upstream binary releases.

If your OS provides packages but the packages are outdated, it's not necessarily a reason to switch (especially if you have to build it yourself). Often outdated versions of packages still work fine; our Ubuntu systems run a lot of outdated versions of things like Exim, Dovecot, and Apache, because the Ubuntu versions are always behind the official releases. What drove us to switch with Prometheus was that the Ubuntu versions being outdated actively mattered. They weren't just outdated, they were buggy and limited.

(Sometimes sticking with OS packages will lead you to skip entire OS releases, because one release has an okay outdated version but a newer one has a broken outdated version. But this can be perfectly okay, as it is for us in the case of Exim and Ubuntu 20.04. But if Ubuntu 22.04 also turns out to have a version of Exim that we don't want to use, we'll have to change course and use an upstream version.)

A related reason is if the upstream strongly recommends against using the OS packages. This is the case with rspamd, where the official site specifically urges you not to use the Debian and Ubuntu packages. Like Prometheus, rspamd provides its own pre-built binaries that are officially supported, so we use those rather than take whatever risks are there with the Ubuntu version. Spam filtering is also one of those fields where the software needs to keep up with the ever changing Internet (spam) landscape in order to be as effective as possible.

(Of course now that I've looked I've discovered that there isn't even an rspamd package for Ubuntu 18.04. But we made the decision based on that being what the upstream strongly recommended, and we're going to stick with it even for Ubuntu releases where Ubuntu does provide an official rspamd package.)

Once you start using an upstream version you have to decide how often to update it. My views here depend on how frequently the upstream does releases, how rapidly they evolve the program, and generally how much trouble you're going to have with catching up later with a whole bunch of changes at once (and how much the upstream believes in backward compatibility). A project with frequent and regular releases, a significant churn in features and options, and a low commitment to long term backward compatibility is one where you really want to keep up. Otherwise you can consider freezing your version, especially if you have to build and update things manually.


Comments on this page:

By Albert at 2020-11-23 05:05:45:

A lot of those issues become non-issues if you run the apps in containers (for example Grafana).

By cks at 2020-11-23 22:47:14:

The short version is that containers only help break any connection between the host OS and the software, they don't take away all of the problems of chosing what version of the software to use, where to get it from, and when to update it (or not update it). I wrote more about this in a new entry here.

Written on 23 November 2020.
« Sometimes it's best to use the official upstream versions of software
What containers do and don't help you with »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Nov 23 00:34:45 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.