The cost of virtualization

June 13, 2008

Virtualization is in the air around here, partly as a way of being environmentally friendly; fewer physical machines means less power and heating and so on. Now, I'd like to be environmentally friendly, but one of the issues with virtualization is that getting into it for anything important has significant startup costs.

One of the reasons we run different services on different machines to start with is for fault isolation, so we don't have all of our eggs in one basket. Virtualization reverses that; if you lose your host machine, you will lose any number of virtual machines. Thus if you are virtualizing important machines you need to be able to recover from this, which means at least two fairly beefy host machines and something approximating a SAN.

(This assumes that you aren't using virtualization just as a way of dealing with stupid software that demands to take over a whole machine.)

This issue makes it hard to just dabble your toes in virtualization. It's a serious investment and thus a serious commitment to get involved, which makes it a hard sell unless you are convinced that it is the way to go.

(This is especially so if you don't already have a SAN environment and a need for some big servers, so that if virtualization doesn't work out in practice you can't just use the hardware for something else.)

Note that in many cases you don't need expensive vendor software for live migration and failover and so on. Certainly in our environment it would be good enough just to bring the storage for the virtual machines up on another host machine and boot them back up, exactly as if we'd had a localized power failure with real hardware.

(Don't laugh. How many machines would you lose right now if one of your rack PDUs failed? If you can accept that, you can accept a virtualized version of it, especially as you can probably recover faster in the virtualized case. You might even be able to automate the recovery.)

Comments on this page:

From at 2008-06-20 09:09:07:

Have you ever run ESX or Xen in HA modes? Your whole argument goes out the window.

By cks at 2008-06-20 13:12:04:

I don't think any HA mode can eliminate the need for multiple relatively powerful servers, although with extra software and disk space you might be able to get away from the need for some sort of SAN itself.

(You need relatively powerful host machines because you need to be able to run all of your virtual machines after you lose one host. This implies that each host has to be over-configured for its usual load, so that after you lose one machine your remaining pool of hosts can take over the orphaned virtual machines without dying.)

To put it another way: if you're getting into virtualization for anything important, you can't just use some existing spare servers you happen to have lying around; you're going to have to build a real environment for the purpose, which costs a not insignificant amount of money. (And it costs more if you have to pay license fees for things like ESX HA support.)

From at 2008-06-21 09:31:16:

You're right, in that the idea of virtualization is a tradeoff.

For example, I just built my infrastructure using a blade enclosure of 10 servers. I could have done the same thing with 2 extra large machines in a high-availability failover setup with ESX. Because I'm only dealing with 10 physical machines, it wouldn't be worth the tradeoff. If I scaled that up an order of magnitude and had the option of building 100 blades or 20 high-availability ESX machines, the extra cost of ESX is such a smaller percentage that it pays me to only have the 20 servers.

Due to the relative higher cost of the 20 large servers, they're highly redundant and less likely to go down due to transient infrastructure issues. You're also more likely to have redundant, well, everything, if you're spending the sort of money to get an infrastructure of servers like that.

After researching it for several months, I'm convinced that in the right infrastructure, Virtualization can have a much higher uptime than a lot of enterprise-level individual server implementations. It's just not for everyone, due to the extra cost of the HA virtualization environment.


Written on 13 June 2008.
« The problem with ZFS, SANs, and failover
Why DNS blocklists return information as IP addresses »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri Jun 13 23:59:26 2008
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.