Back in March I wrote about how ARM servers had better just work, which pointed to various articles on what seemed to be a fairly problematic situation around basic questions like whether there was a standard way to boot Linux on ARM based servers. Today skeeto asked what I'd make of the news of Red Hat Enterprise Linux introducing ARM server support. So I opened my mouth on Twitter and got some very useful replies from Jon Masters, who is Red Hat's chief ARM architect. The short version of the answer is that 'modern' ARM servers (by which I mostly mean future ARM servers, since I don't think very many are out now) will basically behave like x86 servers with UEFI.

ARM itself has established a ServerReady certification program, and it has an underlying standard for server booting called SBBR that goes with a standard for server architecture as a whole (SBSA). The ServerReady program appears to provide a convenient way to know if your ARM-supporting Linux should boot on a particular ARM server; if the ARM server has the certification, it supports everything that Linux needs to boot. Presumably ARM servers will basically all be ServerReady ones. Given the Red Hat announcement, RHEL is going to support ServerReady ARM servers; CentOS either already does or perhaps will soon. Ubuntu already supports ARM systems with UEFI firmware in 16.04 LTS, at least according to their wiki page.

(Other OSes that can use UEFI and ACPI can presumably take advantage of the ARM standards too. FreeBSD is apparently on course to make 64-bit server ARM a fully supported platform, per here.)

All of this sounds very nice, but then I paid attention to a little sentence from Cloudflare's recent post on ARM servers:

Up until now, a major obstacle to the deployment of ARM servers was lack, or weak, support by the majority of the software vendors. In the past two years, ARM’s enablement efforts have paid off, as most Linux distros, as well as most popular libraries support the 64-bit ARM architecture. Driver availability, however, is unclear at that point.

There was enough driver support that Cloudflare could run Linux on an engineering sample of Qualcomm's Centriq server platform, but that still leaves a lot of question marks. If ARM servers have an entirely new set of hardware for things like Ethernet, USB, and SATA, then we could be back in the early dark days of Linux on x86 servers, where every new server came with big question marks about Linux support (especially in 'long term support' stable releases). An optimistic hope would be that ARM server vendors will be reusing a lot of existing PCIE-based hardware and the Linux drivers for it all are architecture-independent and just work on ARM.

(To a certain extent I expect that this has to be the way things will go. It seems unlikely that ARM server vendors will try to do their own hardware for 10G Ethernet, SAS chipsets, computational GPUs, and various other things (or that it would be very successful if they tried).)

PS: even if all the hardware works, the cost challenge remains important for us. As a result I doubt we'll be using ARM servers any time soon, since I expect the early generations of them to be focusing on people who care about total system performance for the cost, performance per watt, and issues with heat and density.