Some NVMe drive temperature things from my drives

December 9, 2021

I said some things on Twitter about the temperatures of the NVMe drives in my home and work machines (as reported by the Linux kernel), so I'm going to write down more on that here. I don't know if it means anything, but it's at least some data points.

Both my work desktop and my home desktop now have a pair of NVMe drives in them, all of which report the nominal drive temperature to the Linux kernel. The machines are built using the same case but have different motherboards, different sorts of CPUs, and different PCIe card layouts (which influences heat flow inside the case). And of course they sit in different environments, so both the ambient exterior temperature and the case interior temperature are likely different.

At home I now have a pair of Crucial P5 NVMe drives, which are reported to run comparatively hot. Both are mounted in motherboard M.2 slots; one of the slots is low on the board, away from the CPU, and covered by a motherboard heatsink, while the other is just below the CPU. Both are unused right now and at idle, they are both consistently around 41 C. When I put them under test load, the one under the heatsink goes up to 53 C, while the other one goes up to 62 C (despite being only using two PCIe lanes instead of four). So my first moderate surprise is that the motherboard M.2 NVMe heatsink actually does seem to really be doing something. Whether it makes a performance difference I don't know, but it clearly makes a heat difference.

(The Crucial P5 NVMe drives have two temperature sensors; one source suggests that one is on the controller and the other on the flash. Apparently the flash is usually lower temperature than the controller. My numbers are for what Linux reports as the 'Composite' temperature. At idle, the controller temperature seems to be 4 to 5 C higher than the composite temperature, while at load the two controllers reach 60 C and 77 C.)

At work I have a pair of Kingston A2000 NVMe drives, which apparently run relatively cool. One is mounted in a motherboard M.2 slot that has no heatsink and is located right between my Radeon RX 550 and the Ryzen CPU; the other is mounted on a PCIe expansion card, also without a heatsink. These are in use for the machine's root filesystem and core ZFS pool, but the machine itself is mostly idle since I'm not at work. The NVMe drive on the PCIe card seems to idle around 30 C, while the one on the motherboard idles around 33 C. Under normal heavier IO load like compiling Firefox, they can get as high as 45 C and 47 C respectively. Under artificial load of full streaming reads from the block devices, they will go to 50 C and 54 C respectively. The moderate surprise here is how much location and perhaps mounting can matter.

All four drives warm up and cool down fairly rapidly. This is probably not really surprising since all of the drives are obviously small and thus presumably don't store much heat. It's still impressive to see a drive at high temperature lose 10 C in a matter of thirty seconds or so once the load goes away.

PS: Right now, both motherboard temperature sensors claim to be reading about 30 C and all of the NVMe drives are at their 'idle' temperature. Where the various motherboard temperature sensors are located and what actually influences them is an open question; it's not like vendors label them on the board for you.

Comments on this page:

By Miksa at 2021-12-09 04:53:45:

There is regular debate about whether NVMe are necessary or beneficial. If the NVMe controller runs too hot it will throttle the performance, but I think this only occurs around 90°C or so, and this usually only happens during synthetic benchmarks. In normal use it's not really possible to generate enough IO.

But the actual flash chips on the other work better when they run hot, I think it reduced degradation or something. I've seen a good sounding explanation for this but I couldn't find it again. This article adresses the issue.

Written on 09 December 2021.
« NVMe drives and the case of opaque bandwidth limits
Why it's good to explicitly document the purposes of things, illustrated »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Dec 9 00:27:12 2021
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.