One of my peculiarities is that I reboot my Fedora 40 desktops by logging in as root on a text terminal and then running 'reboot' (sometimes or often also telling loginctl to terminate any remainders of my login session so that the reboot doesn't stall for irritating lengths of time). Recently, the simple process of logging in as root has been stalling for an alarmingly long time, enough time to make me think something was wrong with the system (it turns out that the stall was probably ten seconds or so, but even a couple of seconds is alarming for your root login not working). Today I hit this again and this time I dug into what was happening, partly because I was able to reproduce it with something other than a root login to reboot the machine.

My first step was to use the excellent extrace to find out what was taking so long, since this can trace all programs run from one top level process and report how long they took (along with the command line arguments). This revealed that the time consuming command was '/usr/libexec/pk-command-not-found compinit -c', and it was being run as part of quite a lot of commands being executed during shell startup. Specifically, Bash, because on Fedora root's login shell is Bash. This was happening because Bash's normal setup will source everything from /etc/profile.d/ in order to set up your new (interactive) Bash setup, and it turns out that there's a lot there. Using 'bash -xl' I was able to determine that pk-command-not-found was probably being run somehow in /usr/share/lmod/lmod/init/bash. If you're as puzzled as I was about that, lmod (also) is apparently a system for setting up paths for accessing Lua 'modules', so it wants to hook into shell startup to set up its environment variables.

It took me a bit of time to understand how the bits fit together, partly because there's no documentation for pk-command-not-found. The first step is that Bash has a feature that allows you to hook into what happens when a command isn't found (cf, see the discussion of the (potential) command_not_found_handle function), and PackageKit is doing this (in the PackageKit-command-not-found Fedora RPM package, which Fedora installs as a standard feature). It turns out that Bash will invoke this handler function not just for commands you run interactively, but also commands that aren't found while Bash is sourcing all of your shell startup. This handler is being triggered in Lmod's init/bash code because said code attempts to run 'compinit -c' to set up completion in zsh so that it can modify zsh's function search path. Compinit is a zsh thing (it's not technically a builtin), so there is no exposed 'compinit' command on the system. Running compinit outside of zsh is a bug; in this case, an expensive bug.

My solution was to remove both PackageKit-command-not-found, because I don't want this slow 'command not found' handling in general, and also the Lmod package, because I don't use Lmod. Because I'm a certain sort of person, I filed Lmod issue #725 to report the issue.

In some testing in a virtual machine, it appears that pk-command-not-found may be so slow only the first time it's invoked. This means that most people with these packages installed may not see or at least realize what's happening, because under normal circumstances they probably log in to Fedora machines graphically, at which point the login stall is hidden in the general graphical environment startup delay that everyone expects to be slow. I'm in the unusual circumstance that my login doesn't use any normal shell, so logging in as root is the first time my desktops will run Bash interactively and trigger pk-command-not-found.

(This elaborates on and cleans up a Fediverse thread I wrote as I poked around.)