What tool you use to read IPMI sensor information can matter
Most modern servers have some sort of onboard IPMI (sometimes called a service processor), and there are good reasons to want to read the IPMI's sensors instead of relying purely on whatever sensor readings you can get from the Linux kernel. There are a number of open source projects for reading IPMI information on Linux (and sometimes other systems); I have direct experience with ipmitool and FreeIPMI, and the Arch wiki lists a few more.
Until very recently I would have told you that these tools were
more or less equivalent for reading IPMI sensors and you could use
whichever one you wanted to (or whichever had the more convenient
output format). Locally we've used
ipmitool for this, and generated
Prometheus metrics by parsing its output. Except that we haven't
been able to this on all of our machines, because on some of our
models of Dell servers, reading the IPMI sensors is achingly slow.
I was all set to write a grumpy entry about the need for reading
IPMI sensors to be fast in order to be really useful when I decided
to test FreeIPMI to see if it would give me better diagnostics
about why reading IPMI sensors on these Dell servers was so slow.
To my surprise, FreeIPMI's
was able to read IPMI sensors with blazing speed on these servers.
This blazing speed is partly the result of building a cache of
sensor data records (SDRs),
but even when
ipmi-sensors has to build its cache it runs several
times faster than
ipmitool (on the Dell server models that are a
ipmitool). Even on Dell server models that
runs decently fast,
ipmi-sensors seems faster.
I have no idea what the two projects are doing differently when they
read IPMI sensors, but there clearly is a real difference in FreeIPMI's
favor, contrary to what I expected. My minimum learning experience from
this is that if I run into IPMI problems when I'm using one project,
ipmitool, I should try other projects just to see. I'm also
probably somewhat too cynical about the quality of implementation of
IPMIs, at least from major server companies; Dell probably wouldn't ship
an IPMI that really took five minutes to read out all of its sensors.
I've got a lot of ingrained habits around using
with some scripting. I'm probably going to switch at least the
scripting to use FreeIPMI (which means installing and testing it
on more servers here). I may not switch my habits, especially right
away, since they're pretty ingrained and ipmitool is convenient in
how everything is right there in one program and one manpage.
PS: I was lucky by already having some familiarity with FreeIPMI,
since I'd used its
program to set some OEM-specific options in the IPMIs for some servers