The meaning of "hysteresis" and how it relates to alerts

August 11, 2021

I try not to mis-use terms when I write entries here on Wandering Thoughts, so when I had an entry where I was planning to use the term "hysteresis", I decided to actually look up its real meaning first. That was a good thing because it turns out its actual definition is not quite what I thought (and I would probably have mis-used it). Let's start with the simple definition from Wikipedia's "hysteresis" entry:

Hysteresis is the dependence of the state of a system on its history. [...]

At first look, this seems pretty divorced from monitoring alerts (the thing I associated it with). We can get a better idea in the section on hysteresis in control systems:

In control systems, hysteresis can be used to filter signals so that the output reacts less rapidly than it otherwise would by taking recent system history into account. For example, a thermostat controlling a heater may switch the heater on when the temperature drops below A, but not turn it off until the temperature rises above B. [...]

There's also hysteresis in electronic circuits:

Often, some amount of hysteresis is intentionally added to an electronic circuit to prevent unwanted rapid switching. [...]

There's a lot of things in system administration where we want to prevent 'unwanted rapid switching', or unwanted rapid triggering of something, or the like. One way to do this is to not switch (or trigger or etc) if we've already done this recently. This means that the current state of the system depends partly on its history (whether there's been a recent switch), which is hysteresis.

As I understand it, hysteresis isn't the only way to mitigate flapping alerts, rapid switching, and similar situations, and whether or not some specific techniques (such as requiring conditions to be true for some time) count as hysteresis is something that isn't always clear to me. On the other hand, maybe even 'average over time' counts as hysteresis, since it does mean that the state of the system (whether or not you're triggering an alert) depends on its history.

Based on the definition, hysteresis as a thing is broader than just preventing unwanted rapid switching. I think it's hysteresis if you trigger an alert for projected space exhaustion based on the recent rate of growth, for example, since the state of the alert depends not just on the current state but the past history of space usage.

My overall conclusion is that I probably don't want to talk about "hysteresis in alert triggers" when I mean keeping alerts from flapping too fast, because on the one hand hysteresis is broader than that and on the other hand it may not really be the right term for what I want to do.

(Also, hysteresis is spelled with a single 'i', not two the way I initially thought it was. That mistake almost made it into the name of this entry, which would have been awkward to fix.)

Written on 11 August 2021.
« The Firefox uMatrix addon is not quite dead (so far)
Prometheus, Alertmanager, and maybe avoiding flapping alerts »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Aug 11 00:23:06 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.