Formatting information output to make it easy to manage

June 27, 2011

Suppose that you are designing systems to be managed and you are taking the straightforward approach of dumping out information when external programs ask for it. As it happens, I have some views on how you should format this information and why.

Monitoring systems have two major concerns when they're dealing with your status output. First, they want to be confident that they understand what you're printing out, ie that they are correctly parsing it. Second, they want to be sure that they understand the meaning of everything you print out. Monitoring systems care about both because a problem with either of them can cause a monitoring system to misinterpret your state.

The attraction of XML or JSON as your output format is not so much that monitoring systems don't have to write a parser, it's that they don't have to worry about whether or not they have parsed your output correctly. They don't have to debug a parser or be confidant that it can completely parse everything that you ever output, even in obscure situations.

The corollary of this is that if you want to roll your own output format, you should make it as simple and regular as possible. Simple regular formats make for easy parsers, and easy parsers are more likely to be correct, complete, and bug-free. Conversely, inventing your own complex structured output format is about the worse thing you can do (you get bonus points for things that appear only rarely).

The other concern is harder. Ultimately, understanding requires complete documentation of what your system outputs and what it means. However, you can make things easier by being consistent, for the same reason as with parsing; it makes it simpler for monitoring systems to be confidant that they haven't missed some obscure corner case in how you will report things.

(For some sorts of information you can also be redundant, for example by always flagging certain sorts of situations with a common marker. In theory this lets a monitoring system extract some information from something that it doesn't fully understand, eg if it sees the 'ERROR' field set it knows that something has gone wrong even if it doesn't fully understand what.)

Written on 27 June 2011.
« Design systems to be managed
How to securely run programs from inside your program on Unix »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jun 27 00:45:57 2011
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.