Wandering Thoughts archives

A realization about code complexity and clarity

In the process of writing yesterday's entry I had one of those obvious in retrospect insights:

We all know (or at least have been told repeatedly) that just because something about the code (such as where a name comes from) is clear to you when you write the code, that doesn't mean it will remain obvious later (or be obvious to other people). One of the core reasons for this is context, specially how much context is in your mind.

When you are writing the code, you generally have the maximum amount of context possible in your mind; you just wrote the code, or at least you are working on it intensively. You will lose at least part of this context over time, possibly a lot of it, and of course other people don't have it to start with or at least don't have very much of it. To use yesterday's issue as an example, when you are writing the code of course you know off the top of your head where a certain function comes from, because you just wrote it. If you're using a rule for what functions are defined where, of course you also remember that rule and how it applies to this particular function. But all of this context is going to fade from your mind over time if you're not actively working on the code.

This means that you need to overshoot on clarity, because your view of clarity is being influenced by the context you have. Just being clear isn't enough; things need to be obvious, maybe glaringly obvious to the you that has maximum context, because that means they have a chance of still being clear to someone who has a lot less context.

(It is my personal opinion that clarity and obviousness are better than trying to write enough introductory comments to recreate some approximation of the context.)

PS: if you are writing code that is intrinsically complex enough to require significant context to work on, it might be worthwhile to explicitly note this at the top of the code, more or less as a warning sign. I wouldn't necessarily try to recreate the context, just have a comment that says 'all of the code and comments here assume deep familiarity with ...' or something similar.

One reason I like having this insight is that it creates a grounding for a lot of things I've had hammered into me about creating overly complex and clever code.

Sending email via a SMTP connection considered harmful

When confronted by the need to send email, many programs decide that the best way to do it is via SMTP to your mail server. As they say, now these programs have two problems.

The problem with having your program send email via SMTP is that SMTP receivers are not always available all of the time. There are a thousand and one reasons why the local SMTP receiver is not answering right at the millisecond that your program is trying to connect to it; perhaps the machine is rebooting, or perhaps a sysadmin is restarting the mailer, or there is a brief network glitch. Or maybe it is just overloaded.

The moment your program sends email via SMTP, it becomes responsible for dealing with this. To be specific, you become responsible for building some sort of system for queueing and retrying delivery of your important email. (It might not be a literal queue of mail, although that is the obvious implementation.)

(One dangerous aspect of sending email via SMTP is that it is very easy for programs to drift into assuming that SMTP is essentially completely reliable, and thus that they don't need to think about what to do if it ever fails. Then one day your SMTP server's power supply dies, it's down for hours, and you're left to go through the application logs to figure out what email should have been sent.)

Some people like spending their time building a queueing system to manage email. Other people know that there are well developed programs that already exist to deal with queues of mail and delivering it to potentially flaky SMTP servers; they are called MTAs, or more simply 'mailers'. On a well configured machine, submission of messages to the local mailer basically never fails unless the machine is exploding in general.

To be clear, I am not advocating that you should run a full blown mail environment on each server. I'm just advocating that you should let the MTA worry about queueing and retrying delivery to your real SMTP server, instead of forcing your program to deal with it.

Usefully, many MTAs have a mode where they accept SMTP from standard input and report results to standard output. With socketpair(), it's not that much work to make the rest of your code think that it's talking to a remote SMTP server when it is actually talking to the local MTA.

FastCGI's encoding mistakes and how not to design a wire protocol

One of the things that FastCGI needs to transfer between the web server and the FastCGI server are a bunch of name/value pairs; this is used, for example, to transfer the traditional CGI environment variables for a request to the FastCGI server. FastCGI being FastCGI, it has defined a binary record for this; each name/value pair is encoded as the length of the name, the length of the value, and then the name and value as bytes (with no terminating zero).

In order to reduce the overhead for short names and/or values, FastCGI allows the length of either to be encoded in either a short one byte form or a four-byte form. To tell them apart the protocol uses the high bit of the first byte (which is the high byte) of each length; if the bit is set, it is a four-byte length (and the high bit is masked out when the length is computed). Unset, and it is a one-byte length. This allows FastCGI to save up to six bytes per name/value pair, reducing the protocol overhead from eight bytes to two and, as the specification is careful to note, allows a FastCGI application to immediately allocate a correctly sized buffer for the pair.

(All of this is in the FastCGI specification, section 3.4.)

It is genuinely hard to count all of the terrible mistakes in this wire encoding.

• this significantly complicates decoding and encoding name/value pairs.

• this multi-way encoding optimizes a non-problem. FastCGI is used either on the local machine (over Unix domain sockets or connections to localhost) or at the worst over local gigabit network links. Neither environment is short on bandwidth compared to web server request rates.

  (The FastCGI protocol specs make high-minded appeals to the X Windows protocol, completely ignoring the vast difference in request rates between an X server and a web server.)

• the four-byte length encoding is completely over-engineered, since it can encode names or values that are up to 2 gigabytes in size. Web servers do not get and do not accept header names or header values that are anywhere near that large.

  (The HTTP 1.1 specification does not appear to restrict the maximum length of header names or header values, so if you care about utter spec compliance even a limit of 2 GBytes is too low.)

• in order to 'immediately allocate' a correct sized buffer for the name and value, you must make four separate reads (conceptually): one for the first byte of the name length, possibly another one to get the next three bytes, one for the first byte of the value length, and finally possibly another one to get the next three bytes.

  (Nor can you read a bunch of bytes into a buffer and then start sorting out the mess. Values are allowed to be empty, so the shortest possible name/value record is only three bytes long.)

  After this you can efficiently make that one memory allocation.

• in practice you never send just one name/value pair; you send a bunch of them at once. If FastCGI wanted to reduce the overhead of sending name/value pairs, it should create a single record that covered a sequence of name/value pairs.

• All of this efficiency ignores the fact that this bit of FastCGI is a stream protocol layered over a (multiplexed) message protocol, so you are not reading straight from the raw network into your single memory allocation. You are already copying data around from hither to yon (from the low-level message to your stream of parameter data) and you may already be doing fragment reassembly in arbitrary places.

• still on the efficiency front, since name/value data is not terminated with null bytes many C-based environments will need to copy it in order to create C style null-terminated strings.

Essentially, everything that FastCGI has done here is a micro-optimization that misses the forest for the trees. Not only is it optimizing things that don't matter, it is the protocol equivalent of carefully polishing your bubble sort routine when you ought to be using quicksort. If you want to optimize a wire protocol in this way, you must take a whole stack view; you must look at the flow of data through the whole decoding process (from the point where you get some bytes from the operating system on up), not just at one small portion of it, and you must look at the protocol overhead of an entire transaction (taking all layers into account).

Of course, all of that is a lot less showy than coming up with a complex encoding scheme for lengths that saves six bytes.

Why I call FastCGI complex and SCGI much simpler

In a comment on this entry, Daniel Martin took issue with with my description of SCGI as much simpler than FastCGI. As it happens, I have a straightforward reason for describing FastCGI this way: it's a protocol that uses encoded binary structures.

Any protocol that uses encoded binary structures requires you to write decoders and encoders, or at least a protocol description for your encoder/decoder generator. This is annoying even with simple fixed-format binary structures; it is an outright pain in the rear if the protocol has multiple structures, structures with variable fields, or complex switching logic (where you don't always have a common prefix that you can do simple switch dispatching on). FastCGI has multiple levels of decoding (where you have structures inside structures) and at least one case of an excessively clever structure encoding.

(The excessively clever structure encoding is worth special note, because in the name of saving six bytes FastCGI has four variations on the same basic structure that are differentiated not by an explicit type field, but by which of the first two bytes have their high bits set. In the process, it re-orders structure fields. I could go on about this particular encoding mistake, but this is not the entry for it.)

Regardless of the inherent simplicity or complexity of the protocol in an abstract sense (ie in terms of what the flow of messages is), using encoded binary structures makes you (much) more complicated than an equivalent protocol that uses a simpler, easier to parse and generate encoding. Since FastCGI uses encoded binary structures and SCGI doesn't, I say that SCGI is much simpler than FastCGI.

(FastCGI is also a more involved protocol than SCGI. I could easily call it over-engineered, in part because it reimplements OS-level concepts on its own and generally does this worse than the OS does.)

Sidebar: one OS-level concept that FastCGI reimplements

FastCGI needs to pass multiple communication channels between the web server and the FastCGI server process. As it happens, Unix OSes have a simple and well supported way of doing this; you open up multiple file descriptors between you and the other end and use each file descriptor for a separate channel. FastCGI would still need to do a little bit of multiplexing to handle standard output and standard error over the same channel, but that is much easier than the multiplexing job that it has taken on for itself.

(Doing this does not require multiple processes; you can perfectly well handle all connections with select() et al in one process. It does give you an easy way of using multiple processes, which can be handy.)