2015-02-04
How our console server setup works
I've mentioned before that we have a central console server machine where all of our serial consoles and other serial things all get centralized, automatically logged, and so on. While I don't think we're doing anything unusual in this area, I've realized that doing decent sized console servers is probably no longer common and so it might be interesting to describe how ours works.
The obvious way to do a (serial) console server is just to build a machine with a bunch of serial ports. This kind of works at small or moderate scale, but once you're talking about thirty or fifty or a hundred or more serial ports, things break down. There are two problems with this; first, you just can't fit that many serial ports into one piece of hardware for sane amounts of money, and second, you can't feasibly run serial lines to everything in remote locations (like master switches in building wiring closets and so on).
The thing that makes it possible to deal with all of this is serial
port to Ethernet concentrators; we use various models of Digi's
Etherlite series,
generally the rack-mountable 16 and 32 port versions. These have
some number of RJ-45 ports which we plug serial connections into
and an Ethernet port over which the system talks to their software
on our console server, where a Digi kernel module turns those
networked serial ports into /dev/... serial port entries that
look just like hardware serial ports.
On the console server we use conserver to manage the serial ports; it logs their traffic, handles actual interactive access to them, and so on. Conserver is probably not the only system for this (and may well not be the best); it's just what we use. It works and you can probably find it packaged for your Linux distribution of choice.
(As far as I know there's nothing that will directly talk the Digi Etherlite protocol so you can cut out the middleman of the fake kernel serial ports. I believe this is partly because the protocol is at least undocumented. It's possible that there are other serial port to Ethernet concentrators with documented protocols and thus direct support in projects like conserver.)
If we need serial ports in a remote location, for example to give access to a switch's console, we put an Etherlite in the location and connect it up. The serial connection to the Etherlite is subject to reasonable length limitations but obviously the network traffic is not. We run Etherlites and similar things over a physically separated and independent management network (described at the end of here).
Sidebar: How you connect serial ports to Etherlites
Etherlites don't have conventional serial ports; instead they use plain RJ-45 ports for higher density (which makes them look like Ethernet switches). Plain RJ-45 is also known as 'Ethernet connectors', so we wire things up using ordinary cat-5 Ethernet cables that plug into RJ45 to DB9 adaptors, which then plug into the servers.
(I don't think we have anything left with full sized serial ports; these day's it's DB9 or nothing. Fortunately servers are still coming with DB9 serial ports.)
I don't think you need to use full bore Ethernet cables for this. We just happen to already have everything we need to make cat-5 Ethernet cables already, so this way we keep everything standardized. To avoid confusion we use a special colour.