Wandering Thoughts archives

2020-05-13

Getting my head around what things aren't comparable in Go

It started with Dave Cheney's Ensmallening Go binaries by prohibiting comparisons (and earlier tweets I saw about this), which talks about a new trick for making Go binaries smaller by getting the Go compiler to not emit some per-type internally generated support function that are used to compare compound types like structs. This is done by deliberately making your struct type incomparable, by including an incomparable field. All of this made me realize that I didn't actually know what things are incomparable in Go.

In the language specification, this is discussed in the section on comparison operators. The specification first runs down a large list of things that are comparable, and how, and then also tells us what was left out:

Slice, map, and function values are not comparable. However, as a special case, a slice, map, or function value may be compared to the predeclared identifier nil. [...]

(This is genuinely helpful. Certain sorts of minimalistic specifications would have left this out, leaving us to cross-reference the total set of types against the list of comparable types to work out what's incomparable.)

It also has an important earlier note about struct values:

  • Struct values are comparable if all their fields are comparable. Two struct values are equal if their corresponding non-blank fields are equal.

Note that this implicitly differentiates between how comparability is determined and how equality is checked. In structs, a blank field may affect whether the struct is comparable at all, but if it is comparable, the field is skipped when actually doing the equality check. This makes sense since one use of blank fields in structs is to create padding and help with alignment, as shown in Struct types.

The next important thing (which is not quite spelled out explicitly in the specification) is that comparability is an abstract idea that's based purely on field types, not on what fields actually exist in memory. Consider the following struct:

type t struct {
  _ [0]byte[]
  a int64
}

A blank zero-size array at the start of a struct occupies no memory and in a sense doesn't exist in the actual concrete struct in memory (if placed elsewhere in the struct it may have effects on alignment and total size in current Go, although I haven't looked for what the specification says about that). You could imagine a world where such nonexistent fields didn't affect comparability; all that mattered was whether the actual fields present in memory were comparable. However, Go doesn't behave this way. Although the blank, zero-sized array of slices doesn't exist in any concrete terms, that it's present as a non-comparable field in the struct is enough for Go to declare the entire struct incomparable.

As a side note, since you can't take the address of functions, there's no way to manufacture a comparable value when starting from a function. If you have a function field in a struct and you want to see which one of a number of possible implementations a particular instance of the struct is using, you're out of luck. All you can do is compare your function fields against nil to see whether they've been set to some implementation or if you should use some sort of default behavior.

(Since you can compare pointers and you can take the address of slice and map variables, you can manufacture comparable values for them. But it's generally not very useful outside of very special cases.)

GoUncomparableThings written at 23:29:19; Add Comment

2020-05-04

The Go compiler has real improvements in new versions (and why)

When I wrote that I think you should generally be using the latest version of Go, I said that one reason was that new versions of Go usually include improvements that speed up your code (implicitly in meaningful ways), not just better things in the standard library. This might raise a few eyebrows, because while it's routine for new releases of C compilers and so on to tout better performance and more optimizations, these rarely result in clearly visible improvements. As it happens, Go is not like that. New major versions of Go (eg 1.13 and 1.14) often provide real and clearly visible improvements for Go programs, so that they run faster, use less memory, and soon will take up somewhat less space on disk.

My impression is that there are two big reasons that this happens in Go but doesn't usually happen in many other languages; they are that Go is still a relatively young language and it has a complex runtime (one that does both concurrency and garbage collection). Generally, Go started out with straightforward implementations of pretty much everything (both in the runtime and in the compiler), and it has been steadily improving them since. Sometimes this is simply in small improvements (especially in code generation, which sees a steady stream of small optimizations) and sometimes this is in much larger rewrites, such as the one that added asynchronous preemption of goroutines in Go 1.14 or the currently ongoing work on a better linker. Go's handling of memory allocation and garbage collection has especially seen a steady stream of improvements, sometimes major ones, such as the set covered in Getting to Go: The Journey of Go's Garbage Collector.

(And back in 2015, there was the rewrite of the compiler to have a new SSA backend (also), which unlocked significant opportunities for additional optimizations since then.)

Generally, other languages have had some combination of having a lot longer to mature and extract all of the straightforward optimizations from their compiler, having a simpler runtime environment that doesn't need as much development effort, or having a lot of very smart people working on them. Java, Javascript, and the Microsoft .NET languages all have complex runtimes, but they also have a lot of resources poured into their implementations, which means that they often improve at a faster rate than Go does (and they all pretty much started earlier). C and C++ compilers generally have simpler runtime environments that need less work and have also had a lot longer to optimize their code generation. What C compilers can already do is pretty spooky, so it's not terribly surprising that the improvements now are mostly small incremental ones. It will likely be a long time before Go gets to that level, if it every does (since there is a tradeoff between how fast you can compile and how much optimization you do, and Go values fast compile times).

GoRealImprovementsWhy written at 00:19:24; Add Comment

By day for May 2020: 4 13; before May; after May.

Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.