Wandering Thoughts archives

Go does not have atomic variables, only atomic access to variables

Suppose, hypothetically, that you have a structure full of expvar variables. You would like to expose all of them in one operation with expvar.Func, using some code that goes roughly like this:

var events struct {
   Var1, Var2, Var3 expvar.Int
}

func ReportStats() interface{} {
   return events
}

Ignoring for the moment how this won't work, let's ask a more fundamental question: is this a safe, race-free operation?

On first blush it looks like it should be, since the expvar types are all safe for concurrent access through their methods. However, this is actually not the case, due to an important thing about Go:

Go does not have atomic variables, only atomic access to variables.

Some languages support special atomic variable types. These variable types are defined so that absolutely all (safe) language access to the variables that you can perform is atomic, even mundane accesses like 'acopy = avar' or 'return avar'. In such a language, ReportStats() would be safe.

Go is not such a language. Go has no atomic variable types; instead, all it has is atomic access to ordinary non-atomic variables (through the sync/atomic package). This means that language level operations like 'acopy = avar' or 'return avar' are not atomic and are not protected against various sorts of data races that create inconsistencies or other dangers. The expvar types are no exception to this; their public methods are concurrency safe (which is achieved in various ways), but the actual underlying unexported fields inside them are not safe if you do things like make copies of them, as ReportStats() does when it says 'return events'.

In some cases you can get a warning about this, as go vet will complain about the related issue of making a copy of a sync lock (or anything containing one, such as a sync.RWMutex). Some types that are intended to be accessed only atomically will have an embedded lock, and so making a copy of them will cause go vet to complain about copying their embedded lock. However, not all 'intended to be atomic' types use embedded locks, so not all will be caught by this check; for example, expvar.String has an embedded lock (in a sync.RWMutex) and so will provoke go vet complaints when copied, but expvar.Int currently doesn't have an embedded lock and go vet will not warn you if you copy one and give yourself a potential data race.

(There may someday be a way to annotate types so that go vet knows to complain about making copies of them, but as far as I know there's no way to do that today. If such a way is added, presumably all expvar variable types would get that annotation.)

A quick trick: using Go structs to create namespaces

Suppose, not entirely hypothetically, that you have a bunch of expvar statistics variables in your program that you're registering yourself in order to create good names for them in the exposed JSON. Implemented normally, this probably leaves you with a bunch of global variables for various things that your program is tracking. Just like any other jumble of global variables, this is unaesthetic and it would be nice if we could do better.

Well, we can, due to Go's support for unnamed struct types. We can use this to basically create a namespace for a collection of variables:

var events struct {
   connections, messages  expvar.Int
   tlsconns, tlserrors    expvar.Int
}

In our code we can now refer to events.connections and so on, instead of having to have more awkward or more ambiguous names.

We aren't restricted to doing this at the global level, either. You can fence off any set of names inside such a namespacing struct. One example is counters embedded into another structure:

type ipMap struct {
   sync.Mutex
   ips   map[string]int
   stats struct {
       Size, Adds, Lookups, Dels  int
   }
}

For obvious reasons, this works best for variable types that don't need to be initialized; otherwise things get at least a least a little bit awkward with how you have to set up the initialization.

This is probably not to everyone's tastes and I don't know if it's considered good Go practice. My personal view is that I would rather fence off variable names with 'prefix.' than with say 'prefix_', even at the cost of introducing such an artificial unnamed struct, but other people probably differ here. It does feel a bit like a hack even to me, but maybe it's a legitimate one.

(For statistics counters specifically, this can also give you a convenient way of exposing them all.)

Out of curiosity I did a quick scan of the current development Go compiler and standard library and turned up a few things here and there that might be this pattern. There are variations and it's not all that common, so the most I'm going to say based on looking is that this doesn't seem like a completely outrageous and unsupported idea.

You can't use expvar.Func to expose a bunch of expvar types

Suppose, hypothetically, that you have a collection of statistics that are each one of the expvar package's types. You want to put them in a namespace (so they are all 'mything.var1', 'mything.var2' and so on), but you'd like to avoid the tedious clutter of registering each expvar variable by hand with .Set(). So you have a clever idea.

First, you will embed all of the variables in a structure:

var somestats struct {
   var1, var2, var3   expvar.Int
   var4, var5, var6   expvar.Int
}

Then you will write a Stats() function that simply hands back the structure and then register it through expvar.Func():

func ReportStats() interface{} {
   return somestats
}

Unfortunately, this will not work (at least today). As I mentioned in my first set of notes on the expvar package, expvar.Func turns what your function returns into JSON by using json.Marshal, and this only returns exported fields. None of the expvar variable types have any exported fields, and so as a result expvar.Func() will convert them all to empty JSON (or possibly malfunction). You just can't get there from here.

This is kind of a bug (at a minimum, expvar.Func should document this restriction), but it's unlikely to change (apart from the documentation being updated). Beyond it not working today, there's no way to have a simple ReportStats function like this that work safely, and since you can't do this there's little to no point in making expvar variable types JSON-serializable through json.Marshal.

(To make this work, each expvar type would implement a MarshalJSON() method that did the appropriate thing. In fact, since expvar.String is really MarshalJSON() in disguise, you could just make one call the other.)

Sidebar: Why clear and complete documentation matters

Here is a question: is it deliberate that the 'thing to JSON' function for expvar.Var is not called MarshalJSON, or is it a historical accident? You can certainly argue that because my pattern above is fundamentally wrong, it's a feature that it doesn't work at all. Thus the choice of not using MarshalJSON in the expvar.Var interface could be entirely deliberate and informed, since it makes a broken thing (and all its variants) simply not work. Or this could be ultimately a mistake on the order of using String() in the expvar.Var interface, and so something that would be corrected in a hypothetical Go 2 (which is allowed to change APIs).

Without better documentation, people who come along to Go later just don't know. If you want to preserve the intent of the original designers of the language and the standard library, it really helps to be clear on what that intent is and perhaps the logic behind it. Otherwise it's hard to tell deliberate decisions from accidents.