When you should run an NTP daemon on your servers

November 17, 2017

Yesterday I made the case for why you mostly shouldn't run an NTP daemon and should instead synchronize time on your servers through periodic use of ntpdate or similar programs. However, mostly is not all of the time and I think that there are times when running an NTP daemon is the right answer.

So here is a list of when I'd run an NTP daemon:

  • Your servers need accurate time, where they're always within a few milliseconds or less of true time (for some definition of true time).

    If you need this, you'll want to carefully design your local NTP setup, including both your server hierarchy and your external time sources (you want stratum 1 time sources, ideally ones where you have stable network paths to them). In extreme cases you'll want to set up your own stratum 1 server based on, for example, GPS time.

  • Your servers need synchronized time, where they're always within a few milliseconds or less of each other.

    The canonical case for tightly synchronized time is a set of NFS fileservers, where clients write files to multiple fileservers and want them all to have the same timestamps (with high precision). Synchronized time is less demanding than accurate time; you just need a NTP server hierarchy where everyone synchronizes to a core set of NTP servers and those core servers get good enough time from the outside world.

    (Another case is synchronized timestamps across multiple machines.)

  • You absolutely can't have time go backwards, even a little bit, even if the server's time is very off. A stronger version is that you can never have even large forward clock jumps; the clock must always slew slowly, even if it takes a long time to adjust to true time.

    Modern versions of ntpdate, sntp, systemd-timesyncd and so on can slew the clock for modest adjustments. However, for large adjustments this takes sufficiently long that you need an always running daemon to supervise and fine-tune the process.

    (Note that the famous Cloudflare incident was not a case of time going backward due to time synchronization and wouldn't have been prevented by running an NTP daemon. If anything, it might have been caused by running one.)

  • You need ongoing monitoring of the clock state on your servers.

    An NTP daemon on each server makes a good way to keep tabs on the clock state of all of them. Querying full NTP time source parameters will give you additional warning markers, such as unusual delays or clock dispersions to your NTP time servers.

  • Your servers have absolutely terrible local clocks that drift rapidly; for example, they might be off by a second in the span of ten minutes. An always-running NTP daemon is a better way of reining this in than running ntpdate or another client every few minutes.

    (Basically the NTP daemon is going to be slewing the clock either frequently or all the time, just to keep it under control.)

  • You have only a few servers, they can talk to the Internet, and you want a hassle-free way of giving them decent time.

    The preferred NTP daemon on modern Unixes generally comes with a sensible default configuration that will get time from some suitable pool of NTP servers out on the Internet. Depending on what pool it's pointed at, you may even get NTP servers that are close to you (in a network sense). Install the daemon, make sure it's enabled, and almost all of the time your servers will wind up with perfectly good time. If you're not sure what the time state of a server is, query the local NTP daemon and it'll tell you.

    (Of course, your Linux may already be doing this with something like systemd-timesyncd, in which case you don't need to do anything.)

(There are probably additional cases I'm not thinking of right now.)

Finally, you may want to have an NTP daemon on one or two (or three) servers to act as the local source of NTP time that everyone synchronizes to through ntpdate cron entries or the like. I don't consider this the same sort of thing as the above cases, because you're using the NTP daemons as infrastructure instead of directly as time sources for servers; that they also maintain time on their host servers is kind of an incidental side effect (although a useful one).

As you might suspect from this list, our fileservers run NTP daemons so that we have coherent, synchronized time across our NFS environment. Nothing else does (apart from the NTP daemons that act as local time sources).


Comments on this page:

In general, my philosophy is that any machine whose logging gets synchronized to a central logging repository needs to be time sync'd, as well.

But then, I also maintain my own time server from a GPS signal, mostly because I can.

By cks at 2017-11-17 11:35:29:

I think that all your machines should be time sync'd. The question for me is how closely. Running an NTP daemon is necessary if the answer is 'within a few milliseconds', but if the answer is 'within a hundredth of a second', you can almost certainly just run ntpdate/sntp/etc from cron every hour. For a central syslog server, well, syslog timestamps are only to the second.

(And with a central syslog server I prefer to have the server add the timestamps, not the clients, because that automatically makes all messages use the same time base no matter what the clients have happen to their clocks.)

By Twirrim at 2017-11-18 18:30:20:

Running an NTP daemon is often very important if your servers are virtualised. The hypervisor compresses and expands the gaps between timer interrupts as befits the expectations of the host operating system as well as the hypervisor process itself. This can cause all sorts of complications with VMs running under the hypervisor.

Even this advice gets conflicted, Canonical says one thing, RedHat another and so on.

Also to add complexity to the mix, KVM has a clock for paravirtualised instances on it, which allow them to sync to the host clock natively: https://rwmj.wordpress.com/tag/kvmclock/

Honestly? It takes so little effort and resources, I'd generally rather run an NTP daemon than not.

Written on 17 November 2017.
« I think you should mostly not run NTP daemons on your machines
AMD Ryzens, their memory speed peculiarities, and ECC »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Nov 17 01:41:42 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.