Why I don't think upgrading servers would save us much power

March 28, 2016

In a comment on this entry, Ben Cotton noted:

The other consideration for upgrading low-utilization servers is power (and thus cooling) efficiency. [...]

Although I haven't gone out and metered this with our current server fleet, my impression is that we wouldn't save very much here, if anything. Of course my impression may be wrong, so it's time to go exploring.

Almost all of our potentially upgradeable servers are basic entry level 1U servers (all except the one or two remaining Dell 2950s). My strong impression is that like 1U server prices and hard disk prices, 1U server power usage has basically stayed flat for the past relatively decent amount of time. What's changed is instead how much computation you got for your N watts.

(To a large extent this isn't too surprising, since a lot of the power usage is driven by the CPUs and at least Intel has been on fixed power budgets for their CPUs for some time. Of course, as time goes on you can get more performance from the lower powered CPUs instead of needing the high-power ones, assuming they're offered in servers you're interested in.)

Am I right about this? Well, the evidence is mixed. Our primary server generations today are SunFire X2100s and X2200s versus Dell R210 IIs and Dell R310 IIs. The SunFires are so old that it's hard to find power usage information online, but various sources seem to suggest that they idle at around 88 watts (and back in 2006 I measured one booting as using 130 watts). A Dell R210 II is claimed to idle at a measured 43.7 watts, which would be about half the power of a SunFire. On the other hand, an R310 seems to idle at almost 80 watts (per here, but that may not match our configuration), very close to the SunFire power at idle.

(I care about power at idle because most of our servers are idling most of the time.)

All of this assumes a one for one server replacement program, where we change an old server into a new, perhaps more power efficient server. Of course if you really want to save power, you need to consolidate servers. You can do this through virtualization or containers, or you can just start co-locating multiple services on one server; in either case, you'll win by converting some number of idle servers (all with their individual baseline idle power draws) into one almost idle server with a power draw hopefully only somewhat higher than a single old server.

Could we do this? Yes, to some extent, but it would take a sea change in our opinions about service isolation and management. Right now we put different services on different physical services both to isolate failures and to make our lives easier by decoupling things like server OS upgrade cycles for different services. Re-joining services (regardless of the means) would require changing this, although things like containers could somewhat deal with the server OS upgrade cycle issue.

Comments on this page:

By watcher at 2016-03-28 11:41:30:

Why are you seriously trying to do this?

New hardware will not only be geared to new technologies but to new operational ways of thinking.

So ofcourse new servers will be geared to visualization and running the most cores on the smallest hardware possible. If you have multiple servers to facilitate redundancy, then you get 2x (or more) Virtual hypervisors going and have 1 run the primary set of Servers with the next Hypervisor running the reserve set of servers. One goes down, the other takes over. You know that this setup uses a minimal amount of power, once you replace your disk drawers with a few SSD's, you will see your power plummet.

Naturally if you want, you can always try something like this in a power source (during an outage window). http://www.ebay.com.au/itm/AU-Plug-Power-Energy-Consumption-Watt-Meter-Electricity-Usage-Test-Equipment-/111674842039?hash=item1a0056dbb7:g:Y8wAAOSwRLZT10MF

By James (trs80) at 2016-03-28 12:16:00:

watcher has a point - modern CPUs have more and more cores, so single-purpose servers are more and more inefficient. Virtualisation will still give you OS-level isolation, and there are free solutions like oVirt and Proxmox that will manage small clusters of VM hosts.

The only downside is doing host upgrades, at which point you have to start thinking about shared storage to do live migration of guests.

Personally I spec my VM host servers with 6 core CPUs at a high clock speed and as much RAM as is feasible, since RAM is the real constraint these days.

Written on 28 March 2016.
« The limits of open source with Illumos and OmniOS
An awkward confession and what we should do about it »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Mar 28 01:29:50 2016
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.