An open question: part uniformity versus unit cost

October 30, 2013

We're in the process of renewing the hardware for our fileserver infrastructure and at this point we've basically settled on the motherboard we'll use for both the iSCSI backends and the fileservers and the hardware for the iSCSI backends; the backends will be built around a 3U case with 16x 3.5" drive bays on the front and 2x 2.5" drive bays on the back (and dual power supplies and various other bits). As we were discussing things today, we wound up asking a question: does it make sense to specify out a different case for the fileservers?

Because they don't need the drive count of iSCSI backends, the fileservers don't need a 3U case; they could certainly fit in a 2U case without problems, even with room for the hack of local L2ARC SSDs. Based on some preliminary investigation I just did, using a 2U case instead of a backend 3U case could save us perhaps $200 (maybe more, maybe less, it's hard to tell). This is not pocket change to us; if nothing else, that would buy that much more RAM for each fileserver.

But there are two advantages of using the current 3U case for fileservers as well as backends. First, it means that we don't have to search out, specify out, price out, and probably get in an initial evaluation unit of a 2U fileserver setup. Instead we can simply buy, right now, as many servers as we want. This points out the second advantage: parts uniformity. If the only hardware difference between backends and fileservers is how much memory they have and maybe what expansion network card they have, we simplify our lives as far as parts and spares go. We're only buying and deploying one thing instead of two and our spares become more flexible and possibly cheaper.

(There's also extra flexibility that we're unlikely to ever use, for example we'd have the ability to build a low(er)-cost ZFS fileserver that uses local disks instead of iSCSI at the cost of no failover and so on.)

I honestly don't know if this parts uniformity is worth the cost. It's attractive, though (partly because I've already spent enough of the past few months trying to get vendors to talk to me and take our money).

(True parts uniformity would require more than just the case, we'd have to throw in an extra SAS card for each fileserver.)

Comments on this page:

By -dsr- at 2013-10-30 08:00:23:

Does rackspace cost you anything? In most environments, part of the cost of a server's lifecycle - sometimes a major chunk - is the rent, real or notional, on the real estate it takes up and the opportunity cost versus another server you could have fit in.

If you can get a 2U chassis that uses the same modular power supplies and disk trays as the 3U, that should provide nearly all the benefits available for parts uniformity.

If you have a good projection of expected hardware death rates, that can tell you how many spares you need to stock. On new hardware, this can be difficult...

Enough per-unit $200 savings buys you another spare system.

By cks at 2013-10-30 08:39:20:

I should have mentioned this: rack space isn't an issue for us at the moment and likely into the future.

On spares, our standard practice is to buy one spare of anything. We assume (sometimes optimistically) that if we need the spare we'll be able to buy another one to be the spare after that, or have the dead unit serviced under its warranty, or so on. Thus with a uniform server, the one spare serves as both a fileserver spare and a backend spare; with non-uniform servers we'd have to buy one spare for each.

dunno how big your dc but i think a good rule of thumb is that if you don't plan to buy a rack's worth of new chassis', stick with the standard you already have.

Written on 30 October 2013.
« If you're on the IPv4 Internet, you really are in public now
Naming disk devices: drive IDs versus drive locations »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Oct 30 00:55:13 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.