We may have reached a point where (new) ARM servers will just work for Linux

November 14, 2017

Back in March I wrote about how ARM servers had better just work, which pointed to various articles on what seemed to be a fairly problematic situation around basic questions like whether there was a standard way to boot Linux on ARM based servers. Today skeeto asked what I'd make of the news of Red Hat Enterprise Linux introducing ARM server support. So I opened my mouth on Twitter and got some very useful replies from Jon Masters, who is Red Hat's chief ARM architect. The short version of the answer is that 'modern' ARM servers (by which I mostly mean future ARM servers, since I don't think very many are out now) will basically behave like x86 servers with UEFI.

ARM itself has established a ServerReady certification program, and it has an underlying standard for server booting called SBBR that goes with a standard for server architecture as a whole (SBSA). The ServerReady program appears to provide a convenient way to know if your ARM-supporting Linux should boot on a particular ARM server; if the ARM server has the certification, it supports everything that Linux needs to boot. Presumably ARM servers will basically all be ServerReady ones. Given the Red Hat announcement, RHEL is going to support ServerReady ARM servers; CentOS either already does or perhaps will soon. Ubuntu already supports ARM systems with UEFI firmware in 16.04 LTS, at least according to their wiki page.

(Other OSes that can use UEFI and ACPI can presumably take advantage of the ARM standards too. FreeBSD is apparently on course to make 64-bit server ARM a fully supported platform, per here.)

All of this sounds very nice, but then I paid attention to a little sentence from Cloudflare's recent post on ARM servers:

Up until now, a major obstacle to the deployment of ARM servers was lack, or weak, support by the majority of the software vendors. In the past two years, ARM’s enablement efforts have paid off, as most Linux distros, as well as most popular libraries support the 64-bit ARM architecture. Driver availability, however, is unclear at that point.

There was enough driver support that Cloudflare could run Linux on an engineering sample of Qualcomm's Centriq server platform, but that still leaves a lot of question marks. If ARM servers have an entirely new set of hardware for things like Ethernet, USB, and SATA, then we could be back in the early dark days of Linux on x86 servers, where every new server came with big question marks about Linux support (especially in 'long term support' stable releases). An optimistic hope would be that ARM server vendors will be reusing a lot of existing PCIE-based hardware and the Linux drivers for it all are architecture-independent and just work on ARM.

(To a certain extent I expect that this has to be the way things will go. It seems unlikely that ARM server vendors will try to do their own hardware for 10G Ethernet, SAS chipsets, computational GPUs, and various other things (or that it would be very successful if they tried).)

PS: even if all the hardware works, the cost challenge remains important for us. As a result I doubt we'll be using ARM servers any time soon, since I expect the early generations of them to be focusing on people who care about total system performance for the cost, performance per watt, and issues with heat and density.


Comments on this page:

What use cases would an ARM server solve?

By cks at 2017-11-14 08:36:30:

Cloudflare's usage case seems to be performance per dollar and performance per watt on heavily parallel workloads, which are reasonable concerns for people in a datacenter who stress their machines. However, our primarily interest has always been cheaper small servers (or what you could call 'servers per dollar'); we mostly don't have large performance needs and we're not constrained by power or cooling.

It's possible that ARM will be so successful in the server space that it'll drive Intel (and AMD) entirely out of the low-end server market, but that doesn't really seem likely. Intel likes the market and they have the volume to push CPU and SoC prices down. Much of the rest of the costs are the same between ARM and Intel servers; regardless of the CPU, you still need the sheet metal for the case, the power supply, the PCB for the motherboard, and so on.

I'd definitely like ARM to be competitive in the server market, for various reasons. It'll be nice to have the world not be an all-x86 place again.

By skeeto at 2017-11-14 10:21:59:

Thanks! This is exactly the sort of response I had hoped to spur with my question. :-) I'm glad to see things are moving in a positive direction.

I've been using small, cheap, single-board ARM computers (Odroid, Raspi, etc.) as small servers for years. They have HDMI ports and can display a graphical desktop, but I disable it and never use it. These little servers are silent and cool, so it's as if they don't even physically exist. That easily beats my previous practice of repurposing old x86 desktops as small servers.

Written on 14 November 2017.
« X11 PseudoColor displays could have multiple hardware colormaps
I've switched from ntpd to chrony as my NTP daemon »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Tue Nov 14 00:56:14 2017
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.