Wiring offices for sysadmins

December 24, 2011

Our office full of sysadmins has a network wiring problem: we don't have enough. Watching how we've dealt with this problem has given me some opinions on how you should wire an office area for sysadmins in specific, as opposed to just general usage.

In a conventionally wired office area, all of the drops (network ports) run back to a big wiring closet (generally one to a floor or so) or even all the way back to your machine room. In the wiring closet or machine room, the drops go to a patch panel and are then connected to appropriate switches (and reconnected, as your networking needs change). This is a perfectly sensible arrangement and has the great advantage that you don't need to go into office spaces in order to shuffle what network a port is connected to; assuming that you have an accurate port number you can just go to the wiring area, switch the cabling, and you're done.

However, this is not the right setup for a sysadmin office area. In a sysadmin office area all of the drops should go to a wiring closet area in the office itself, which is also where all of the connections from the main wiring closet or machine room should go. Why?

A sysadmin office area has the unusual requirement that we periodically want to set up new private networks, ones that are mostly or completely disconnected from our regular networks. Going off to the machine room or the floor's wiring closet every time you want to do this is a time-consuming pain; since sysadmins are either lazy or very good at working efficiently (depending on your perspective), the end result is that most of the ad-hoc testing networks will actually be implemented by just running wires around the office. The end result of this is wires strung all over the place.

(The exception is any test network that needs to touch servers in the machine room.)

Running sysadmin drops back to something in the office makes it easy to set up these ad-hoc testing networks, in fact easier than grabbing some cabling. This is what you want to keep the office in some sort of order.

There are various downsides to this two-stage wiring, with different ones depending on how you've set things up. The top level summary is that, well, you've added another wiring closet and thus another level of indirection in your network. My personal opinion is that it's worth it. If you want to reduce the problems, you could wire the normal office drops straight through to the normal wiring point and then add extra drops (clearly marked) that go only to the in-office wiring area. The drawback of this is that you have to decide how many of each sort of drops each spot will need instead of being able to adjust the purpose of drops on the fly.

Comments on this page:

From at 2011-12-25 06:35:24:

you know VLANs? I prefere one 24port "real" switch for us sysadmins including a few dump 8port gbit ones (one per vlan for prototyping). Each sysadmin box is on the "real" switch with bonded and trunked Gbit link and can choose to receive every VLAN in the room (make sure to not open all vlans of your network ;).

the only downside of VLAN trunks is bandwidth if you copy e.g between two servers on two different VLANs so you only get ~60MB/s. But if your admins really need Gbit you can just bond them and be done to have native speed on two VLANs (as we done mostly).

By cks at 2011-12-25 12:24:10:

We're quite familiar with VLANs and have a lot of them in our environment. See WiringForSysadminsII for reasons why I don't think a single VLAN'd switch is the solution.

(While bonding overcomes some of the bandwidth problems of using a single VLAN'd switch I don't think it overcomes all of them, and of course the other problems are still there. My view is that bonding only scales so far and it leaves you with uncertainties about how the bonding works exactly, what the achievable bandwidth really is, and so on.)

From at 2011-12-25 13:21:42:

Upstream bandwidth isn't really an issue... If we need more than 10G we'd just use a second uplink for the big sysadmin switch.

Yea either you constantly modify the core switches or let everything through. But imho the real problem is really the need to have many different security seperated vlans/networks in our sysadmin pcs.

Our solution is not ideal though but for this we just use some easy lightning red patch cables and statically configured small gbit switches (most of them not from our big sysadmin switch but directly patched to the entry-points of those networks). No sysadmin should connect any box permanently to those switches. And trust me those policies are really hard to enforce even among us. And yes for your sysadmin PC you'd need at least 4 NIC ports for every possible combination.

For iscsi we couldn't find a better solution as to just get the vlans on the big switch as we'd need bandwidth more than anything on those networks. -- gebi

From at 2011-12-25 13:23:04:

Maybe OpenFlow will provide a solution to all this mess someday... -- gebi

Written on 24 December 2011.
« Disk space in the modern world
Why office switches plus VLANs aren't the answer for sysadmins »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sat Dec 24 01:07:21 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.