Any KVM over IP systems need to be on secure networks

March 26, 2020

In response to my entry wishing we had more servers with KVM over IP (among other things) now that we're working remotely, Ruben Greg raised a very important issue in a comment:

KVM over IP: Isnt this a huge security risk? Especially given the rare updates or poor security of these devices.

This is a very important issue if you're using KVM over IP, for two reasons. To start with, most KVM over IP implementations have turned out to have serious security issues, both in their web interface and often in their IPMI implementations. And beyond their generally terrible firmware security record, gaining access to a server's KVM over IP using stolen passwords or whatever generally gives an attacker full control over the server. It's almost as bad as letting them into your machine room so they can sit in front of the server and use its physical console.

(The KVM over IP and IPMI management systems are generally just little Linux servers running very old and very outdated software and kernels. Often it's not very good software either, and not necessarily developed with security as a high priority.)

If you use either KVM over IP or basic IPMI, you very much need to put the management interfaces on their own locked down network (or networks), and restrict (and guard) access to that network carefully. It's very much not safe to just give your KoI interfaces some extra IPs on your regular network, unless you already have a very high level of trust in everyone who has access to that network. How you implement these restrictions will depend on your local networking setup (and on how system administrators work), and also on how your specific KVM over IP systems do their magic for console access and so on.

(I know that some KVM over IP systems want to make additional TCP connections between the management interface and your machine, and they don't necessarily document what ports they use and so on. As a result, I wouldn't try to do this with just SSH port forwarding unless I had a lot of time to devote to trying to make it work. It's possible that modern HTML5-based KVM over IP systems have gotten much better about this; I haven't checked our recent SuperMicro servers (which are a few years old by now anyway).)

PS: This security issue is why you should very much prefer KVM over IP (and IPMI) setups that use a dedicated management port, not ones that share a management port with the normal host. The problem with the latter is that an attacker who has root level access to the host can always put the host on your otherwise secure KVM over IP management network through the shared port, and then go attack your other KVM over IP systems.

Comments on this page:

By Ruben Greg at 2020-03-26 13:09:34:

Thanks for the post.

I for one, (working at a Uni - we have no colocations) would prefer to get devices without IPMI totally (not always possible). Our research groups buy whatever they want (usually maximum permissible to use all the budget -they are sold all these as great additions by sales).

Recently, a research group replaced all HDDs with SSD and somehow swapped the SATA connectors - so they wanted to change boot priority - entered UEFI/BIOS - could not find appropriate knobs to tweak. Especially some poweredge's behave funky in bios when few PCIe cards are installed. So I told them over phone to reset to defaults. All went on good after that.

Result: IPMI was open to all until a smart PhD student informed us a week later.

Written on 26 March 2020.
« The problem of your (our) external mail gateway using internal DNS views
OpenBSD's 'spinning' CPU time category »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Mar 26 01:34:08 2020
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.