Go's API stability and making assumptions, even in semi-official code

November 27, 2023

Right now, there are some Go programs, such as errcheck, that will work if you build them with Go 1.21 but fail if you build them with the latest Go development version. In fact they will panic deep in the Go standard library, specifically in go/types. Given Go's promises about compatibility, you might expect that this is an error in the development version of Go, especially since the path to the panic goes through golang.org/x/tools, which is also written by the Go developers (although it's not stable and its own API isn't covered by compatibility guarantees). However, it's not a Go problem (cf, also). Instead, it shows either how tricky API compatibility is in practice or alternately how almost anyone can fall prey to Hyrum's law (obligatory xkcd).

The problem is actually more or less in black and white in the code of golang.org/x/tools, although I had to stare at the diff in the change request for a while in order to see it. The relevant old code of golang.org/x/tools was:

// types.SizesFor always returns nil or a *types.StdSizes.
response.dr.Sizes, _ = sizes.(*types.StdSizes)

I must inform you that in the Go development version, the assertion in the comment is no longer true. As a result, the type assertion here usually or always fails, resulting in response.dr.Sizes being nil all the time, and then you later get a panic downstream.

(If the type assertion was a single argument one, it would panic. But since it's the two argument form, it's returning a nil, stored in response.dr.Sizes, and a boolean false that gets thrown away. I believe the error case was handled separately earlier, so in practice response.dr.Sizes was expected to be non-nil.)

However, this is not an API break in Go because types.SizesFor() has never promised to return a types.Sizes interface object with a specific underlying type, including a *types.StdSizes. It just happened to always do so until relatively recently, and originally golang.org/x/tools was written assuming that (undocumented) behavior instead of restricting itself to the public API promises.

The current version of golang.org/x/tools has been updated to properly handle this in CL 516917. But to use the fixed version, other projects need to update their Go module version of golang.org/x/tools to the recent one (and possibly deal with any API changes that matter to them). Since golang.org/x/tools is still a major version 0 module, you can't even blame Go's minimum version selection algorithm for this; it's never even theoretically safe to update a thing on major version, except maybe between patch levels.

Sidebar: The contributing design decision

The specific panic in go/types happened because the methods of types.*StdSizes had an implicit requirement that they not be called on a nil pointer value. When you have methods on a pointer to a type ('pointer methods'), you always have to decide how to handle a nil pointer. Sometimes you work, sometimes you explicitly check for the nil and return errors, and sometimes you let Go panic because this is outside your API (or you've chosen an API without error returns on everything). This decision isn't necessarily well documented.

(By Hyrum's law, if your API doesn't document anything either way and works on nil pointer values in some version, changing it so that it does panic in a later version is at least an implicit API change. It's probably not much of an API change in practice if the results for a nil pointer value were unusable garbage.)

Written on 27 November 2023.
« The HTML viewport mess
Why we scrape Prometheus Blackbox's metrics endpoint »

Page tools: View Source, Add Comment.
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Mon Nov 27 23:30:42 2023
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.