In praise of KVM-over-IP systems

December 5, 2012

We're in the process of migrating to a new generation of server hardware and it is, for me, a kind of a sad moment. You see, while the new servers have generally better specifications, the old servers have one big thing that the new servers don't; a built in KVM-over-IP system as part of their remote management capabilities.

On the surface, perhaps KVM-over-IP ought not to be a big deal; all it really saves you is an occasional trip down to the machine room and we're not supposed to be that lazy. But this is the wrong way to look at it. What KVM-over-IP really means is that you can (re)install servers while doing other things.

When installing servers requires a trip to wherever the server is, it's an interruption and it means that you basically have to drop everything else you're doing to trek down to the machine room and babysit the server (don't forget the right install media, either). Interruptions are a pain, unproductive time is a pain, and sitting in a noisy, cold machine room is a pain too, so there's an incentive to avoid the whole thing as much as possible. All of this is friction.

KVM-over-IP systems remove this friction. You don't need to stop working on anything else, you don't need to leave your office, and you don't need all of the minutiae of installs (media, keyboard and display and mouse, a chair if you're going to be there long, a piece of paper with vital details about the machine, etc etc). If your installs take twenty minutes with fifteen minutes of non-interaction where you're just waiting for it to finish, no problems, this is just like any other fifteen minute process that sits in a window in the corner of your display until it's done. And if you need things during the install for some reason, you have the full resources of your usual working environment.

(A lot of this also applies to any other sort of testing or troubleshooting that needs console access to the machine. As long as you don't need to change hardware itself, you can do everything from your desk with your full working environment available to help; you don't have to stop everything else to relocate to the machine room so you can babysit the machine's console.)

Or in short, a KVM-over-IP system goes a long way towards making real servers just as convenient to deal with as virtualized ones on your desktop. And that's pretty convenient, when you get down to it.

PS: some people's answer to this is 'oh, I'll install the server in my office'. Given how noisy modern servers are, this is generally not going to be popular with people around you even if you can stand the noise yourself. It also doesn't help if the server is already set up down in the machine room and you're reinstalling it to, for example, repurpose it or upgrade its OS.

As a side note, where KVM-over-IP really shines is when you have several machines to (re)install at once, especially if you're trying to do it with as short a downtime as feasible. With KVM-over-IP, installing multiple machines just means a few more windows on your display.

(In addition to my experiences with our new servers, this entry was inspired by reading the start of this blog post (via). Yes, the wheels of blogging grind very slowly sometimes.)

Sidebar: remote power cycling and so on

I'm assuming that KVM-over-IP also includes 'remote media', where you can use ISO images on your desktop as (virtual) CD or DVD drives on the server. This seems to be a general feature these days.

I'm not including remote power cycling as a KVM-over-IP benefit because you can get that in a lot of ways. Particularly, most servers these days have a basic IPMI management interface that supports it.

KVM-over-IP is highly useful in certain circumstances, for example if a machine has problems, needs to have its console inspected, and everyone's at home or whatever. But I'm assuming that the answer to that one without a KVM-over-IP system is either a shrug or calling a taxi and anyways, that sort of thing is generally rare.


Comments on this page:

From 46.144.78.131 at 2012-12-05 06:51:11:

don't your new servers have some sort of LOM (http://en.wikipedia.org/wiki/Lights_out_management, like hp ilo, dell drac, etc)?

Because that's what we use to achieve what you are describing.

What kind of servers are you buying ;-)? a server without a LOM card is not a server IMO. Even my hp microserver at home has one of those so I don't need to go wherever the thing is to get to the console.

From 128.210.189.101 at 2012-12-05 09:28:32:

I assume when Chris says "KVM-over-IP", he's referring to things like iLO, DRAC, etc. I agree with everything Chris says. It's not about laziness, it's about efficiency. In my previous job, we supported over 4000 servers and compute nodes, most in buildings halfway across campus. The ability to get a console without having to take a 15 minute walk made troubleshooting and OS upgrades much easier.

From 63.166.216.16 at 2012-12-05 10:40:28:

To achieve this, I always configure my servers for serial line consoles, either to actual serial lines connected to a Cyclades port server kind of device, or to "virtual" ones provided by the ALOM/iLO/DRAC built in. Even at current disk prices, recording video is prohibitively expensive, whereas console ASCII(ish) is quite affordable.

Downside: sometimes part of the serial line stack has important pieces missing (like the IBM RAID BIOS that needed a "DEL" character to remove an array, but couldn't recognize an ASCII DEL character - only a USB or PS/2 keyboard version).

Upside: I've been surprised by what goes to the console that gets missed by the logging system.

The topic is discussed in Limoncelli and Hogan's "The Practice of Systemn and Network Administration", section 17.1.10, "Console Servers".

By cks at 2012-12-05 11:59:14:

Our new servers have IPMI and other basic LOM stuff but not actual KVM-over-IP (at least not without an extra-cost module, which means that we don't buy it).

The problem with serial consoles is that there's a fair amount you can't do over them, especially when installing a machine. Even when you have an installer environment that will work okay over a serial console you don't have remote media so you have to visit the machine room to deal with that.

(Note that there are drawbacks to LOM on a server. To start with, in practice it usually consumes one of the server's network ports since you can't connect the LOM to anything except a highly restricted high-trust network.)

From 67.180.195.209 at 2012-12-05 19:09:09:

Many new server class systems have IPMI which should perform the functions you are looking for. This will give you at minimum a serial console over lan which will allow you to configure the bios, raid controller, and network interface. Many server or motherboard manufactures refer to this functionality as a BMC controller (Baseboard Management Controller). Usually the ipmi service just needs to be enabled and assigned to a port on the server. Some motherboards even support sharing a single port for both OS use and BMC/IPMI functions via a virtual switch.

IPMI setup information for a dell: http://helvick.blogspot.com/2010/08/ipmi-serial-over-lan.html

Some IPMI implementations also have virtual media and virtual java console functions. One such implementation is available on supermicro systems: http://www.supermicro.com/products/nfo/ipmi.cfm

The IPMI implementations I have worked with (Sun, Dell, and HP) allow full OS pass through to the serial interface allowing a linux/unix virtual serial console to be available over the ip serial port. As far as the OS is concerned the virtual serial port the IPMI controller presents is just a plain old serial port. Thus serial redirection can be configured. All mainstream unix/linux oses have supported serial interface redirection for over two decades. Usually, a line just needs to be added to the kernel boot paramaters to enable serial redirection.

On a slightly different note if you are reloading more then a couple systems on a regular basis I highly recommend setting up a PXE deployment environment. The IPMI serial over lan connection is more than sufficient to connect to a PXE boot server and pick an image to load on the server. PXE deployment combined with a configuration management service such as puppet or chef should allow you to fully configure a server for an end user in under 20 min. That includes base os install and additional component configuration (apache, postgres, security lockdown, etc).

If you are deploying windows servers it appears that microsoft does have some ipmi serial console redirection built into the kernel. This can at least be used for WMI functions or as an emergency console to get RDP functioning. http://msdn.microsoft.com/en-us/library/windows/hardware/ff542282(v=vs.85).aspx

If you are using workstation class hardware you may also have BMC features hidding in the bios. I think Intel may have rebranded the functionality as a management type feature.

Also, if your servers lack BMC/IPMI/iLO/etc but still have a serial port you can configure the serial port to be an output device for the os and sometimes the bios post portion.

Written on 05 December 2012.
« You can't assume that your performance problems will be obvious
One way to break down how people use virtualization »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Wed Dec 5 02:42:40 2012
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.