Wandering Thoughts archives

Fedora co-mingles its source packages with Red Hat Enterprise Linux

In the RPM package management world, the source RPM is what packages are built from and the source RPM's specfile is what describes how to build the binary RPMs (in addition to other package metadata). For a specific and relevant example, the Fedora specfile for the kmod RPM is what says (or used to say) that the weak-modules script should be added to the binary RPM, although it's not in the upstream source and this created a DKMS problem.

However, this is a somewhat incomplete and misleading description of the real problem. The underlying issue was that calling it 'the Fedora kmod specfile' is not really accurate. In reality, the Fedora source RPM for the kmod RPM (and thus the specfile) was also the (future) Red Hat Enterprise Linux source RPM and specfile for kmod. The weak-modules script comes from RHEL and is (probably) useful there, but because Fedora uses the same specfile, it was blindly shipped in Fedora as well. The actual fix to remove weak-modules doesn't remove it from the specfile entirely but adds more complexity to make it conditional.

(Since it's specifically excluded only on Fedora, a Fedora-derived distribution will have to modify this specfile. If the distribution notices, and if it doesn't already pretend to be Fedora for package building purposes, since this probably isn't the only such specfile.)

This kmod issue illustrates at least one potential problem with this dual usage of specfiles. Fedora and Red Hat Enterprise Linux are not the same thing, so there are undoubtedly other places where RHEL packaging needs diverge from Fedora packaging ones. At the moment it's up to package maintainers to manually identify such places and deal with them in some way. If they don't, one or the other of Fedora and RHEL will have issues. If people are lucky, those will be immediate and obvious and will be fixed. Otherwise, you get something like my problem.

This situation is unfortunately probably unavoidable because in practice, Fedora is not an independent distribution; instead it's an arm of Red Hat Enterprise Linux. Red Hat (now IBM) originated what became Fedora, uses Fedora as the base for RHEL, significantly funds Fedora, and still hosts important resources for it (Fedora bugs are reported to bugzilla.redhat.com, for example). Red Hat has no interest in spending the effort to maintain two independent sets of source RPMs and RPM specfiles.

(This situation isn't the same as Debian and Ubuntu. Ubuntu uses Debian's packaging work, but Ubuntu's changes for their own needs don't automatically wind up in Debian.)

PS: I believe it's relatively rare that the latest RHEL and Fedora will have the same version of a package, so it's probably usually not the case that the source RPM is the same on both. Instead, the RHEL package most likely started off as some older version of the Fedora package, so we can informally say that they use the same source RPM and specfile.

Simple use of Let's Encrypt on OpenBSD is pleasantly straightforward (as of 6.8)

For reasons beyond the scope of this entry, I recently needed to get a Let's Encrypt TLS certificate for testing on an OpenBSD machine, which isn't something I've done before. On a relatively modern OpenBSD (6.8), it was pleasantly straightforward and easy, with the programs necessary already installed in a full base install (which is what we normally do on our OpenBSD machines, since a full install is so small).

OpenBSD's standard Let's Encrypt client is acme-client, which has to be configured through /etc/acme-client.conf and then invoked (for example) as 'acme-client -v yourhost' to start the process of getting your TLS certificate. As the OpenBSD documentation tells you, a sample acme-client.conf is in /etc/examples and is easy to edit into shape to list the names you want Let's Encrypt certificates for. I opted to add the optional contact option to the 'letsencrypt' authority in my acme-client.conf, although in retrospect it's pointless for a test server where I don't care if the certificate expires after I'm done.

In the OpenBSD tradition, acme-client is very minimal. Unlike more all encompassing Let's Encrypt clients like Certbot, acme-client doesn't do either post-renewal actions or answering the HTTP challenges.

(Certbot and many other heavy-weight clients have their own little internal HTTP server that they'll run for you for the duration of a challenge, if you ask. This is Certbot's standalone mode.)

To handle the HTTP side of things, the easiest approach is to run OpenBSD's standard httpd server at least temporarily. OpenBSD ships with a sample httpd.conf in /etc/examples that I was able to use with very few changes. Because I wanted to be able to test my new certificate, I left the HTTPS version of my host in my httpd.conf (although it wasn't serving anything), but you could remove it to have a HTTP server that's just there to answer the Let's Encrypt challenges. Pleasantly, OpenBSD httpd will still start if you have a HTTPS site configured but the TLS certificates for it are missing. This lets you leave your HTTPS site configured before you've gotten the Let's Encrypt certificate for it.

(The default OpenBSD httpd.conf redirects all of your HTTP site to your theoretical HTTPS site, which nicely makes it serve nothing except answers to Let's Encrypt challenges.)

Because I was getting my Let's Encrypt TLS certificate for something other than serving a web site, I didn't permanently enable the HTTP server. I then had to start httpd more forcefully than usual, with 'rcctl -f start httpd'; otherwise it reported:

/etc/rc.d/httpd: need -f to force start since httpd_flags=NO

(This is unlike the Prometheis host agent package, which can be started temporarily with 'rcctl start node_exporter' before you decide to permanently enable it. This is probably routine to regular OpenBSD users.)

Once I'd started httpd, acme-client succeeded without problems. Unlike Certbot and some other clients, acme-client has no separate Let's Encrypt account registration process; it registers an account if and when necessary to get your certificate. I opted to get both the certificate ('domain certificate ...' in acme-client.conf) and the default full chain ('domain full chain certificate ...'). This isn't strictly necessary, since you can always manually extract each individual certificate from the full chain file, but I wasn't sure how I was going to use the certificate so I opted to save myself time.

(Since this was a quick test setup, I haven't tried to automate any renewal, but the acme-client manpage has an example cron entry. You need a separate cron entry for every separate certificate you have on the machine; unlike Certbot, there is no 'try to renew anything necessary', even though all of your certificates are listed in your acme-client.conf.)