A consistent preference in APIs for kernel and other low-level APIs

July 29, 2013

One of the things that API Design Matters made me think about is how programmers seem to have a distinct preference in interfaces to low-level APIs: almost all of the time, people very much seem to prefer interfaces that are as close as possible to the actual underlying API. The article's criticism of the .NET socket select() API is spot on, yet you'll find very similar versions of select() (with very similar issues) in any number of high level languages. Almost always they are about as close to a straightforward mapping to the underlying low-level select() API as you can get. Go goes the other way; it doesn't expose a select() at all, instead forcing you to do everything through a very different API.

As I mentioned, I think that this is a real preference on the part of many programmers instead of simply lack of imagination on the part of language implementors. Although I have no firm knowledge, I can see reasons why programmers would like it this way; the obvious one is that it's much easier to reason about what your code is really doing if you can see a straightforward mapping between what you're doing and what the low-level API is. Adding magic transformations for 'convenience' may not be considered worth the loss of transparency, especially if you're deliberately using a low-level API precisely because you want fine control over exactly what your code does.

When enough programmers start feeling this way there are obvious social forces steering implementors away from making even minor improvements to APIs. Sure, you could switch from lists to sets for arguments and return results instead of overwriting things in place, but people will give you hassle for it because your API is not a pure mapping any more. It must be tempting to decide that the probably minor usability improvements are not worth the hassle. If you're going to irritate people anyways, you might as well go for a radical change (as Go's net package does).

(This goes with the constraints shaping kernel APIs in the first place, of course.)

Sidebar: why many implementors probably don't go for radical changes

Put simply, API design is hard and doing it well takes a bunch of extra work. As a language implementor you already have enough work simply creating a standard library for your new language even if you don't change the underlying API except to fit it into your language. Plus, if you don't change the underlying API any flaws in it are not really your fault since you're just providing a direct mapping. If you do design a new API, any flaws in that API are your fault and you'll be blamed for any insufficiencies or problems (and you may get people clamouring for access to the underlying API anyways).

(Plus, you may not be enough of an expert in the problem domain to design a good API in the first place. It's not a great idea to demand that language implementors also be experts in everything that the standard library and language environment will connect to.)

On the whole I think a language implementor has to have a lot of confidence both in their understanding of a problem area and in their ability to do API design in order to implement anything other than a straightforward mapping to the underlying API. The implementors of Go stand out precisely because they do have plenty of experience designing networking APIs in addition to implementing languages.

Comments on this page:

From at 2013-07-29 11:37:20:

If you do design a new API, any flaws in that API are your fault and you'll be blamed for any insufficiencies or problems

I think that's the key. If you reproduce known flaws, at least those flaws are already known as are patterns for working around them (although there might not be a lot of .NET developers used to using select() in c).

If you create a new set of flaws, it will take time for the community to find them and to agree on recommended workarounds. And that community is probably smaller by definition than the ones that use c-imitating select implementations, which means fewer resources to work on those workarounds.

From at 2013-07-29 14:28:22:

Documentation effort may also be a consideration. If you only lightly wrap underlying APIs you can refer people to existing documentation.

Written on 29 July 2013.
« The constraints shaping kernel APIs
Phish spam and outside events »

Page tools: View Source, View Normal, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Jul 29 00:49:32 2013
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.