We've been looking into getting some new servers (partly because a number of our existing Dell R210 IIs are starting to fail). Although we haven't run into this yet ourselves, one of the things we've heard in the process of this investigation is that various lines of basic servers are trying to move to M.2 NVMe system disks instead of 3.5" or 2.5" disks. People are generally unhappy about this, for a number of reasons including that these are not fancy hot-swappable M.2 NVMe, just basic motherboard plug-in M.2.

(Not all basic servers have hot swap drive bays even as it stands, though. Our Dell R210 IIs have fixed disks, for example, and we didn't consider that a fatal flaw at the time.)

My first knee-jerk reaction was that server vendors were doing this in order to get better markups from more expensive M.2 NVMe drives (and I just assumed that M.2 NVMe drives were more expensive than 2.5" SATA SSDs, which doesn't seem to always be the case). However, now I think they have a somewhat different profit focused motive, which is to lower their manufacturing costs (they may or may not lower their prices; certainly they would like to pocket the cost savings as profit).

As far as I know, basic motherboard M.2 NVMe is fairly straightforward on a parts basis. I believe that you break out some PCIe lanes that the chipset already supplies to a new connector with some additional small mounting hardware, and that's about it. The M.2 NVMe drive PCB that you've received as a complete part then plugs straight into that motherboard connector during server assembly.

(If the chipset doesn't have the spare two sets of x4 PCIe lanes (or maybe one set), you don't try to make the server an M.2 NVMe one.)

By contrast, 2.5" or 3.5" drives require more physical stuff and work inside the server. Each drive bay needs a power cable and a SATA or SAS cable (which go into their own set of motherboard connectors), and then you need some physical mounting hardware in the server itself, either as hot swap drive bays on the front or internal mounts. During physical enclosure design you'll have extra objects in the way of the airflow through the server and you'll need to figure out the wiring harness setup. During server assembly you'll have extra work to wire up the extra wiring hardness, mount whatever drives people have bought into the drive bays, and put the drive bays in the machine.

All of this is probably not a huge extra cost, especially at scale. But it's an extra cost, and I suspect that server vendors (especially of inexpensive basic servers) are all for getting rid of it, whether or not their customers really like or want M.2 NVMe system disks. If M.2 NVMe drives end up being the least expensive SSD drive form factor, as I suspect will happen, server vendors get another cost savings (or opportunity to undercut other server vendors on pricing).