'Conditional restart' in init.d scripts can be dangerous

November 28, 2009

Yesterday, the lighttpd instance that I run on my workstation was effectively down for about twelve hours; while the daemon was running, it was using the wrong configuration file and so it wasn't really serving anything. In turn, this happened because I installed a lighttpd package update, and as part of the post-update actions the package did '/etc/init.d/lighttpd condrestart'.

In theory, conditional restart in an init.d script will only restart things if the init script has started the daemon in the first place. This is subtly different from 'if the daemon is running', which is what many init.d scripts implement, and what happened to me illustrates the importance of that difference. I don't start lighttpd with /etc/init.d/lighttpd, I start it with a different init.d script that points it to my local configuration file, so when the normal init.d script 'restarted' lighttpd, the new version was running with the system configuration file and thus not doing much.

I can't blame lighttpd and its init script for this problem; it's relying on standard functions provided by the Fedora init.d environment. And I can't really blame Fedora's init.d environment, because the problem is subtle and reasonably difficult to do completely correctly (and I've seen the same problem on other Linuxes). But regardless of where any fault is or isn't, the underlying issue is that 'condrestart' and similar features are dangerously fragile.

The only way to fix this and make conditional restart reliable is to make the daemons restart themselves; on some signal, any running copy of the daemon arranges to re-exec itself with appropriate command line arguments, environments, and so on. Then the init.d condrestart action simply sends this signal to all copies of the daemon that are currently running and lets them sort it all out.

(As a bonus you will have arranged to fix any copies of the daemon that are running, regardless of how they got started, which is probably what you really want to do.)

If you do not do this, please create an officially supported and documented way of changing all of the command line parameters that your init.d script uses to start the daemon, or as a minimum changing the configuration file.

(Note that this being official is important, because that means that I can count on it not breaking over updates.)

Written on 28 November 2009.
« Modern version control systems change your directory layouts
In security, you need to stop the root mistake »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Nov 28 01:48:12 2009
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.