Wandering Thoughts archives

A consistent preference in APIs for kernel and other low-level APIs

One of the things that API Design Matters made me think about is how programmers seem to have a distinct preference in interfaces to low-level APIs: almost all of the time, people very much seem to prefer interfaces that are as close as possible to the actual underlying API. The article's criticism of the .NET socket select() API is spot on, yet you'll find very similar versions of select() (with very similar issues) in any number of high level languages. Almost always they are about as close to a straightforward mapping to the underlying low-level select() API as you can get. Go goes the other way; it doesn't expose a select() at all, instead forcing you to do everything through a very different API.

As I mentioned, I think that this is a real preference on the part of many programmers instead of simply lack of imagination on the part of language implementors. Although I have no firm knowledge, I can see reasons why programmers would like it this way; the obvious one is that it's much easier to reason about what your code is really doing if you can see a straightforward mapping between what you're doing and what the low-level API is. Adding magic transformations for 'convenience' may not be considered worth the loss of transparency, especially if you're deliberately using a low-level API precisely because you want fine control over exactly what your code does.

When enough programmers start feeling this way there are obvious social forces steering implementors away from making even minor improvements to APIs. Sure, you could switch from lists to sets for arguments and return results instead of overwriting things in place, but people will give you hassle for it because your API is not a pure mapping any more. It must be tempting to decide that the probably minor usability improvements are not worth the hassle. If you're going to irritate people anyways, you might as well go for a radical change (as Go's net package does).

(This goes with the constraints shaping kernel APIs in the first place, of course.)

Sidebar: why many implementors probably don't go for radical changes

Put simply, API design is hard and doing it well takes a bunch of extra work. As a language implementor you already have enough work simply creating a standard library for your new language even if you don't change the underlying API except to fit it into your language. Plus, if you don't change the underlying API any flaws in it are not really your fault since you're just providing a direct mapping. If you do design a new API, any flaws in that API are your fault and you'll be blamed for any insufficiencies or problems (and you may get people clamouring for access to the underlying API anyways).

(Plus, you may not be enough of an expert in the problem domain to design a good API in the first place. It's not a great idea to demand that language implementors also be experts in everything that the standard library and language environment will connect to.)

On the whole I think a language implementor has to have a lot of confidence both in their understanding of a problem area and in their ability to do API design in order to implement anything other than a straightforward mapping to the underlying API. The implementors of Go stand out precisely because they do have plenty of experience designing networking APIs in addition to implementing languages.