In universities, you often buy hardware when you can

October 7, 2010

I like to dream that in a real company, you can go out and just buy server hardware when it's needed. Around here, sadly, it doesn't work that way.

At least in my section of the university, one of the problems is that money for hardware can be what I'll call 'bursty'. We don't have an ongoing hardware budget; instead, we usually get one-time money sort of at random, when funds can be found or liberated from grant funds. Over the long term this averages out acceptably, but it's quite possible for there to be dry spells in the medium term for various reasons.

(Oh, if there is a real need or an emergency, money can be found. But even if there is a real need it may take a bunch of time to get it and irritate a bunch of important people in the process.)

One of the consequences of this is that we tend to buy hardware whenever we have the budget for it if we have any plausible future need for it, even if we don't specifically need the hardware right now. Otherwise, when we actually need it in six months or a year or even more we may find that there is no budget for it and we're stuck (barring a real emergency), or at least we have to deal with various sorts of awkward delays.

The downside of buying hardware early, even necessary hardware, is that we'd usually get a better deal if we waited; computer hardware generally drops in price (and improves) over time. But from our perspective it's much better to have hardware in reserve so that we can roll out capacity expansions smoothly as they're needed even if it results in us getting an objectively worse deal on, eg, hard disks.

(Note that we are generally not doing things that demand leading edge CPU performance or the like, which is one reason this is a viable thing to do. Most of our servers are years old anyways and work just fine for pretty much everything we deal with. Research groups doing heavy computation have much different needs and thus much different buying patterns.)

There are some aspects of this that we could handle better. For example, given that a certain amount of the hardware we buy will sit around unused for significant amounts of time, we should probably make a habit of unboxing everything when it comes in and at least turning it on to make sure it is not dead on arrival. Otherwise we'll someday open a box and be embarrassed to discover that the hardware is DOA and also old enough that the warranty has already expired.

Sidebar: the other reason to buy when you can

The other reason to buy hardware when you can is that it may not be available any more by the time that you decide you want more. Even for normal hardware this can be annoying, because you have to go through a whole evaluation and qualification process to find a replacement. If you actively liked the hardware in question it's even more irritation. And of course you wind up with a mixed environment instead of a uniform one, which complicates your life in various ways.

(Even if you have no formal requirement for competitive hardware purchasing, you can't buy new hardware and deploy it to production without testing it to make sure that it works.)

As a result, we have a tendency to load up on hardware we really like when we find some (especially if the vendor runs an end of quarter bargain sale; we love those, at least when we have the spare budget). We also like building a good spares pool for things that we're likely to expand in the future, such as network switches or the components of our fileserver environment.

(Network switches are one of those things where you will hate yourself a lot if you wind up with fifty different models and varieties.)

Comments on this page:

From at 2010-10-07 01:52:32:

Even in companies, buying hardware when you can is not unheard of, especially when you're mostly doing "projects" (which means: you buy hardware at the start of the project and see what happens).

From at 2010-10-07 08:10:56:

This, in my environment, was the big sell behind virtualization.

I actually negotiated with other groups to get the hardware behind our VMware campus cluster. Our web group had ordered a bunch of servers that were completely over spec. Brand new IBM x3550 M2 machines with two quad-core Xeons and more memory than they would ever need. I had plenty of extra SAN disk, so I asked them: if I could prove to you with hard data that you don't need the capacity, and I could use the new VMware environment to provide you with a higher-availability solution versus what you have, would you let me throw 48 GB of RAM and a pair of HBAs in those servers and virtualize what's running on them? A few months later, here I am with a substantial six-server cluster, and most of our old IBM blades that nobody can afford to replace are all going away. It's a good thing to carry a good rapport with people in a position to help you.

Decoupling the OS instance from the hardware was one of the best decisions we made because of the difficulty in procurement. Now we've dropped the requirement for a small application from "we need an old server that will probably be dead in a few months" to "we need 1GB of RAM and 10GB of disk somewhere on this vacant cluster."

Are you talking about my University?

In the previous refresh of our F5's, a pair of 6400s were purchased ... and because it would result in a jump from 4.x to 9.x, management felt that we needed to training before touching these (and training had been bundled into the purchase)...but there wasn't time. Though they were telling 'customers' that features would not be available until we upgraded to the new units.

There had been no training involved related to current pair. (and after I had gotten training, I was able to figure out how to implement some of the requests in 4.x...and, the day I got back from training I wasn't just given access but, they became 'mine'. As the first person to have taken training, and stayed ... the year before a co-worker had gone to training, where he caught a job offer and gave notice on his return.)

Anyways..the new 6400s had sat around in their boxes for over a year before the new manager broke the seal ... to find that one of the units was DOA. If it had been turned on before the 1 year mark, it could be replaced under warranty. And, it needs to turn on and register/activate for support....

Hopefully we'll do better with the 7200v's ...

Written on 07 October 2010.
« What is going on with Samba's POSIX locking on NFS on Linux
Why I am unhappy with Upstart right now »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Oct 7 00:02:54 2010
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.