Wandering Thoughts archives

A caution about cgo's error returns for errno

Go's cgo system for calling C functions offers a very convenient feature. As the documentation puts it:

Any C function (even void functions) may be called in a multiple assignment context to retrieve both the return value (if any) and the C errno variable as an error [...]

Reading this, you may be tempted to write more or less standard Go error-handling code like the following:

kcid, err := C.kstat_chain_update(t.kc)
if err != nil {
   return err
}

This code is a potential mistake. Unless the documentation for the C function you're calling says so explicitly, there is no guarantee that errno is zero on success. If the function returns success but errno is non-zero, cgo will dutifully generate a non-nil error return from it and then your Go code will bail out with an error that isn't.

This is not cgo's fault. Cgo has no magic knowledge of what C function return values are and aren't errors, so all it can do is exactly what it said it was going to do; if errno is non zero, you get an error version of it. This is just a C API issue (that ultimately comes about because errno is both an implicit return and global state). You'd never write code like this in Go, where 'only return non-nil error on actual errors' is well established, but we're stuck with the C API that we actually have instead of the Go-like one we'd like. So we have to deal with it, which means checking return values explicitly.

(In this case the real 'there has been an error' marker is a kcid return value of -1. I actually hit an irregular test failure when my code was just checking err, which is how I re-stubbed my toe on this particular C API issue.)

PS: the ultimate cause of this is that C code often doesn't explicitly set errno to zero on success but instead leaves it alone, which means errno can wind up set from whatever internal system call or library routine last failed and set it. There are many possibilities for how this can happen; a classical one is seeing ENOTTY from something checking to see if the file descriptor it is writing to is a TTY and so should be in line-buffered mode.

(In my case I saw EAGAIN, which I believe was the OmniOS kernel telling kstat_chain_update() that the buffer it had been given wasn't large enough, please try again with a bigger one.)

Tweaking code when I'm faced with the urge to replace it entirely

The one of the core parts of DWiki (the program behind all of what you're reading) is the code that turns DWikiText into HTML (and in many ways it is the most important component, since it's what ultimately renders all my content and all the comments). I spent a significant chunk of today tweaking a test version in an attempt to improve the conversion process for a number of corner cases that I care about and would like to make better.

There are two problems with this. The first is that one of the consequences of having what is now a long-running block is that I have a lot of content written in my wikitext dialect and pretty much all of it had better keep coming out just the same. I have no interest in trying to go through all of my entries to revise them for some bright new wikitext idea; backwards compatibility is quite important, warts and all. This is unfortunate in practice because I made some mistakes in my wikitext dialect way back when. It's sometimes possible to do little things around the corner of these mistakes, but that creates hacks and special rules and special magic code.

The other problem is that DWiki the program is also old and by now rather tangled, especially in the DWikiText renderer. Part of this tangle is just history, part of it is that it has been heavily optimized for speed, and part of it is that I made some fundamental structural mistakes in the beginning that have been carried through ever since then (one of them is not parsing to an AST, but it's not the only one). Faced with a complex set of code that I don't work with regularly, I descend to tweaking as carefully as I can rather doing anything deeper, which of course builds up the accumulated layers of hacks in the code and makes it harder to do anything except more tweaks and hacks around the edges.

(Python makes it surprisingly easy to do this sort of tweaking for various reasons, including its support for optional function arguments.)

At the same time, the more I worked on this code today the more clearly I saw how I wanted a modern version of the code to work. The more I have to stick in hacks and make tweaks, the more I also want to raze the whole complicated mess to the ground and redo it from scratch with a much better restructured version (or at least what I think would be such a thing, in my current state of not having written it and thus not having been faced by any messy gaps in my current grand vision).

(Any grand rewrite immediately starts to run into Python 3 thoughts, which lead to other thoughts I'm not going to try to cover here.)

I don't have any answers and I'm not even sure I'm going to deploy the tweaked version (I have a history of this sort of indecision), although I probably will now that I've written this up. But at least I had a reasonably enjoyable time fiddling around in the depths of DWiki once again and perhaps a bit more impetus towards doing some significant cleanups someday.

(While in the past I've lamented that I don't have a test suite for DWiki, I do actually have one for this sort of change; I can render all of this thing in both the old and new versions of the code and see what's different. This can be an interesting (re)learning experience, but that's another entry.)