Timing durations better in Python (most of the time)

November 24, 2019

As I mentioned in yesterday's entry on timeouts and exceptions, we have some Python programs that check to make sure various things are working, such as that we can log in to our IMAP servers. Since they're generating Prometheus metrics as part of this, one of the metrics they generate is how long it took.

(My feeling is that if you're going to be generating metrics about something, you should always include timing information just on general principles even if you don't have an immediate use for it.)

Until recently, my code generated this duration information in the obvious way:

stime = time.time()
do_some_check(....)
dur = time.time() - stime

There is nothing significant wrong with this and most of the time it generates completely accurate timing information. However, it has a little drawback, which is that it's using the current time of day (in general, the clock time). If the system's time gets adjusted during the check, the difference between the starting time of day and the ending time of day doesn't accurately reflect the true duration of the check.

A generally better way to measure durations is to use monotonic time, obtained through time.monotonic(), which the operating system promises can never go backwards and isn't affected by changes in the system's clock. Most of the time the two are going to be the same or very close (sufficiently close that if you're using Python, you probably don't care about the timing difference). But sometimes monotonic time will give you a true answer when clock time will be noticeably off.

The one limitation of monotonic time is that in some environments, monotonic time doesn't advance when the system is suspended, as might happen with a laptop or a virtual machine. Monotonic time is the time as the operating system experiences it, but not necessarily absolute time passing. If you need absolute time passing and you care about the system being suspended, you may have to use clock time and hope for the best.

(In my environment this code is running on a server that should never suspend or pause, so monotonic time is perfect.)

PS: time.monotonic dates back to Python 3.5 or before, so it should be pretty safe to use everywhere (fortunately for us, Ubuntu 16.04 is just recent enough to have Python 3.5). People still using Python 2 are out of luck; consider it another reason to write new code in Python 3 instead.


Comments on this page:

From 193.219.181.211 at 2019-11-24 06:53:28:

Sounds like you may sometimes want CLOCK_BOOTTIME – it is monotonic but does get forwarded after normal PC suspend/resume (although I guess not necessarily after VM suspend, if the OS is unaware of it).

There's no separate function for it in Python but you can use clock_gettime() to access it on any recent kernel.

Written on 24 November 2019.
« Thinking about timeouts and exceptions in Python
I use unit tests partly to verify that something works in the first place »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Nov 24 01:19:08 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.