In yesterday's entry on our trouble free remote reboot of our machines, I mentioned that remote power control for all of our machines would be nice. When I said that, I was insufficiently specific, because there are actually two forms of remote power control and we mostly have one of them, but it's not always good enough. I will call these two sorts of remote power control external and internal remote power control.

In external remote power control, your servers and machines are plugged in to a smart power bar or smart rack PDU that you can remotely control to turn outlets on or off. In internal remote power control, the machine itself has some form of lights out management that will control the machine's power; for instance, power control over (networked) IPMI, or a serial connection to the management process. External remote power control is much easier to set up and is what we have, but it's not quite as good as internal remote power control overall.

The largest limitation of external remote power control is that it leaves you vulnerable to BIOSes with undesirable power control settings. If you have a machine with its BIOS set to 'when power returns after a power loss, stay turned off', then you can't force reboot the machine with external remote power control; once you turn the power to it off, it won't come back up again until you go in to push its physical power button. The machine has power, but the BIOS has been told to keep it off until you push that button.

(Plain KVM over IP will usually not get you out of this, because the BIOS is not powering up the keyboard and video. Wake on LAN probably would, but if your BIOS is set to keep the power off, you probably didn't set WoL up either.)

Plain external remote power control also doesn't do anything if you have the machine plugged into a UPS (either completely, for all of its power supplies, or partially, with one redundant PSU plugged into a UPS and the other into your smart PDU). If you can remote control outlets on the UPS, you can deal with this, but I'm not sure how many UPSes (even rack ones) support this. I don't believe ours do.

(Our fileservers have dual PSUs with one on a UPS and one on line power this way.)

Internal remote power control deals with both issues because it can directly control the machine's PSUs and will simulate you pushing that front panel power button to overcome BIOS settings. The drawback of having only internal remote power control is that you can't completely cut power to the machine, including to its lights out management processor (you can generally tell the LOM to reboot, but not to turn itself off). So the best of both worlds is to have both smart PDUs and lights out management on your machines.