Wandering Thoughts archives

2017-09-12

The different contexts of stopping a Unix daemon or service

Most Unix init systems have a single way of stopping a daemon or a service, and on the surface this feels correct. And mostly it is, and mostly it works. However, I've recently come around to believing that this is a mistake and an over-generalization. I now believe that there are three different contexts and you may well want to stop things somewhat differently in each, depending on the daemon or service. This is especially the case if the daemon spawns multiple and somewhat independent processes as part of its operation, but it can happen in other situations as well, such as the daemon handling relatively long-running requests. To make this concrete I'm going to use the case of cron and long-running cron jobs, as well as Apache (or the web server of your choice).

The first context of stopping a daemon is a service restart, for example if the package management system is installing an updated version. Here you often don't want to abruptly stop everything the daemon is running. In the case of cron, you probably don't want a daemon restart to kill and perhaps restart all currently running cron jobs; for Apache, you probably want to let current requests complete, although this depends on what you're doing with Apache and how you have it configured.

The second context is taking down the service with no intention to restart it in the near future. You're stopping Apache for a while, or perhaps shutting down cron during a piece of delicate system maintenance, or even turning off the SSH daemon. Here you're much more likely to want running cron jobs, web requests, and even SSH logins to shut down, although you may want the init system to give them some grace time. This may actually be two contexts, one where you want a relatively graceful stop versus one where you really want an emergency shutdown with everything screeching to an immediate halt.

The third context is stopping the service during system shutdown. Here you unambiguously want everything involved with the daemon to stop, because everything on the system has to stop sooner or later. You almost always want everything associated with the daemon to stop as a group, more or less at the same time; among other reasons this keeps shutdown ordering sensible. If you need Apache to shut down before some backend service, you likely don't want lingering Apache sub-processes hanging around just because their request is taking a while to finish. Or at a minimum you don't want Apache to be considered 'down' for shutdown ordering until the last little bits die off.

As we see here, the first and the third context can easily conflict with each other; what you want for service restart can be the complete opposite of what you want during system shutdown. And an emergency service stop might mean you want an even more abrupt halt than you do during system shutdown. In hindsight, trying to treat all of these different contexts the same is over-generalization. The only time when they're all the same is when you have a simple single-process daemon, at which point there's only ever one version of shutting down the daemon; if the daemon process isn't running, that's it.

(As you might suspect, these thoughts are fallout from our Ubuntu shutdown problems.)

PS: While not all init systems are supervisory, almost all of them include some broad idea of how services are stopped as well as how they're started. System V init is an example of a passive init system that still has a distinct and well defined process for shutting down services. The one exception that I know of is original BSD, where there was no real concept of 'shutting down the system' as a process; instead reboot simply terminated all processes on the spot.

ThreeTypesOfServiceStop written at 01:12:41; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.