Wandering Thoughts archives

2018-09-28

Addressable values in Go (and unaddressable ones too)

One of the tricky concepts in Go is 'addressable values', which show up in various places in the Go specification. To understand them better, let's pull all of the scattered references to them together in one place. The best place to start is with the specification of what they are, which is covered in Address operators:

For an operand x of type T, the address operation &x generates a pointer of type *T to x. The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x may also be a (possibly parenthesized) composite literal. [...]

There are a number of important things that are not addressable. For example, values in a map and the return values from function and method calls are not addressable. The following are all errors:

&m["key"]
&afunc()
&t.method()

The return value of a function only becomes addressable when put into a variable:

v := afunc()
&v

(Really, it is the variable that is addressable and we have merely used a short variable declaration to initialize it with the return value.)

Since field selection and array indexing require that their structure or array also be addressable, you also can't take the address of a field or an array element from a return value. The following are errors:

&afunc().field
&afunc()[0]

In both cases, you must save the return value in a variable first, just as you need to do if you want to use & on the entire return value. However, the general rule about pointer dereferencing means that this works if the function returns a pointer to a structure or an array. To make your life more confusing the syntax in the caller is exactly the same, so whether or not '&afunc().field' is valid depends on whether afunc() returns a pointer to a structure or the structure itself.

(If you have code that does '&afunc().field', this means that you can get fun and hard to follow errors if someone decides to change afunc()'s return type from 'pointer to structure' to 'structure'. Current Go reports only 'cannot take the address of afunc().field', which fails to explain why.)

Functions themselves are not addressable, so this is an error:

func ifunc() int { return 1 }
&ifunc

Functions point out a difference between Go and C, which is that Go has the concept of pointer-like things that are not pointers as the language sees them. You cannot apply & to functions, but you can assign them to variables; however, the resulting type is not formally a pointer:

v := ifunc
fmt.Printf("%T\n", v)

The type printed here is 'func() int', not a pointer type. Of course you can now take the address of v, at which point you have the type '*func() int'.

By itself, taking the address of things just creates pointers that you can dereference either explicitly with * or implicitly with selectors. The larger importance of addressable things in Go shows up where else they get used in the language.

The most important place that addressability shows up in the specification is in Assignments:

Each left-hand side operand must be addressable, a map index expression, or (for = assignments only), the blank identifier. [...]

(Map index expressions must be specially added here because they aren't addressable.)

In other words, addressability is the primary definition of what can be assigned to. It covers all forms of assignment, including assignment in for statements with range clauses and ++ and -- statements. This implies that if you widen the definition of addressability in Go, by default you widen what can be assigned to; it will include your newly widened stuff.

Because structure fields and array indexes require their containing thing to be addressable, you cannot directly assign to fields or array elements in structures or arrays returned by functions. Both of these are errors:

sfunc().field = ....
afunc()[0] = ....

However, because pointer indirection is addressable, a function that returns a pointer to a structure (instead of a structure) can be used this way:

spfunc().field = ...

This is syntactically legal but may not actually make sense, unless you have another reference to the underlying structure somewhere.

Because all slice index expressions are addressable, if you have a function that returns a slice, you can directly assign to a portion of the returned slice without capturing the slice in a variable:

slicefunc()[:5] = ...

If the slice and its backing array are newly generated by slicefunc() and are otherwise unreferenced, this will quietly discard everything afterward through garbage collection; the only point of your assignment is that it might panic under some situations.

Since map index expressions are a special exception to the addressability requirements, you can assign to an index in a map that's just been returned by a function:

mfunc()["key"] = ...

This has the same potential problem as with the slice-returning function above.

Next, slice expressions sometimes require addressability:

[...] If the sliced operand is an array, it must be addressable and the result of the slice operation is a slice with the same element type as the array. [...]

Taking slices of strings, existing slices, or pointers to arrays does not require that the value you are slicing be addressable; only actual arrays are special. This is the root cause of Dave Cheney's pop quiz that I recently wrote about, because it means that if a function returns an actual array, you cannot immediately slice its return value; you must assign the return value to a variable first in order to make it addressable.

Given that you can take slices of unaddressable slices, just not of unaddressable arrays, it seems pretty likely that this decision is a deliberate pragmatic one to avoid requiring Go to silently materialize heap storage for cases like arrays that are return values. If you do this through a variable, it is at least more explicit that you have a full return object sitting around and that the slice is not necessarily making most of it go away.

Finally, we have Method calls (at the bottom of Calls) and their twins Method values. Both will do one special thing with addressable values:

[...] If x is addressable and &x's method set contains m, x.m() is shorthand for (&x).m(): [...]

And:

As with method calls, a reference to a non-interface method with a pointer receiver using an addressable value will automatically take the address of that value: t.Mp is equivalent to (&t).Mp.

Because return values are not addressable by themselves, you cannot chain method calls from a non-pointer return value to a pointer receiver method. In other words, you can't do this:

// T.Mv() returns a T, not a *T
// and we have a *T.Mp() method
t.Mv().Mp()

This is equivalent to '(&t.Mv()).Mp()' and as we've seen, '&t.Mv()' isn't allowed in general. To make this work, you have to assign the return value to a variable x and then do x.Mp(); at that point, 'x' is addressable and so the implicit '&x' is valid.

The error message you currently get here for an unaddressable value is a little bit unusual, although it may change in the future to be clearer. If you have 'valfunc().Mp()', you get two error messages for this (reporting the same location):

[...]: cannot call pointer method on valfunc()
[...]: cannot take the address of valfunc()

(Gccgo 8.1.1 reports 'method requires a pointer receiver', which is basically the same thing but doesn't tell you why you aren't getting the automatic pointer receiver that you expected.)

Note that the method Mp() is not part of the method set of T; it's merely accessible and callable if you have an addressable value of T. This is different from the method sets of pointer receiver types, *T, which always include all the methods of their underlying value receiver type T.

Intuitively, Go's requirement of addressability here is reasonable. Pointer receiver methods are generally used to mutate their receiver, but if Go called a method here, the mutation would be immediately lost because the actual value is only a temporary thing since it's not being captured by any variable.

(As usual, doing the work to write this entry has given me a much clearer understanding of this corner of Go, which is one reason I wrote it. Addressability uncovers some corners that I may actively run into someday, especially around how return values are not immediately usable for everything.)

PS: Given its role in determining what can be assigned to, addressability in Go is somewhat similar to lvalues in C. However, for various reasons C's definition of lvalues has to be much more complicated than Go's addressability. I expect that Go's simplicity here is deliberate and by design, since Go's creators were what you could call 'thoroughly familiar with C'.

Sidebar: The reflect package and addressability

Addressability is also used in the reflect package, where a number of operations that mirror Go language features also have requirements for addressability. The obvious example is Value.Addr, which mirrors &, and has addressability requirements explained in Value.CanAddr. Note that Value.CanAddr's requirements are stricter than those of &'s:

A value is addressable [only] if it is an element of a slice, an element of an addressable array, a field of an addressable struct, or the result of dereferencing a pointer.

In order to get an addressable reflect.Value outside of a slice, you must pass in a pointer and then dereference the pointer with Value.Elem:

var s struct { i int }
v1 := reflect.ValueOf(s)
v2 := reflect.ValueOf(&s)
v3 := v2.Elem()
fmt.Println(v1.CanAddr(), v2.CanAddr(), v3.CanAddr())

This prints 'false false true'. Only the pointer that has been dereferenced through the reflect package is addressable, although s itself is addressable in the language and v1 has the same reflect.Type as v3.

In thinking about this, I can see the package's logic here. By requiring you to start with a pointer, reflect insures that what you're manipulating through it has an outside existence. When you call reflect.ValueOf(s), what you really get is a copy of s. If reflect allowed you to address and change that copy, you might well wind up confused about why s itself never changed or various other aspects of the situation.

(The rule of converting a non-pointer value to any interface is that Go makes a copy of the value and then the resulting interface value refers to the copy. You can see this demonstrated here.)

GoAddressableValues written at 23:07:59; Add Comment


Page tools: See As Normal.
Search:
Login: Password:
Atom Syndication: Recent Pages, Recent Comments.

This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.