== Inside a Go 'terrible hack' in the _reflect_ package

Recently [[John Allspaw tweeted
https://twitter.com/allspaw/status/653886537445343233]] about some
'terrible hack' comments in the source code of various important
projects. One of them was Go, which made me curious about the
context. Although I can't exactly match Allspaw's version of the
comment, I think it's [[this comment in src/reflect/value.go
https://github.com/golang/go/blob/4c2465d47d8c706832bbc57668680a3ffc4d800f/src/reflect/value.go#L2082]],
which I'm going to quote in full:

.pn prewrap on
 func ValueOf(i interface{}) Value {
     [...]
     // TODO(rsc): Eliminate this terrible hack.
     // In the call to unpackEface, i.typ doesn't escape,
     // and i.word is an integer.  So it looks like
     // i doesn't escape.  But really it does,
     // because i.word is actually a pointer.
     escapes(i)

     return unpackEface(i)
 }

In a nutshell, what's going on here is that the compiler is being
too smart about [[escape analysis
https://en.wikipedia.org/wiki/Escape_analysis]] and the 'hack' code
here is defeating that smartness in order to avoid memory errors.

Per [[Russ Cox's explanation on how interfaces are implemented
http://research.swtch.com/interfaces]], an _interface{}_ value is
represented in memory as essentially two pointers, one to the
underlying type and one to the actual value. _unpackEface()_ magically
turns this into a [[_reflect.Value_
https://golang.org/pkg/reflect/#Value]], which has exactly this
information (plus some internal stuff). Unfortunately it does so
in a way that causes the compiler's escape analysis to think that
nothing from the '_i_' argument outlives ('escapes') _unpackEface()_,
which would normally mean that the compiler thinks '_i_' doesn't
outlive _ValueOf()_ either.

So let's imagine that you write:

>  type astruct struct { ... }
>
>  func toval() reflect.Value {
>     var danger astruct
>     return reflect.ValueOf(&danger)
>  }

Without the hack, escape analysis could tell the Go compiler that
_&danger_ doesn't escape _reflect.ValueOf()_, which would make
_danger_ safe to allocate on the stack, where it would get (implicitly)
destroyed when _toval()_ returns. Unfortunately the _Value_ returned
by _toval()_ actually refers to this now-destroyed stack memory.
Whoops. By explicitly defeating escape analysis, _ValueOf()_ forces
_danger_ to be allocated in the heap where it will outlive _toval()_
and thus avoid this bug.

(You might wonder if Go garbage collection has similar problems and
the answer is apparently 'no', although the details are well beyond
both me and the scope of this entry. See [[this golang-nuts thread
on garbage collection and _unsafe.Pointer_
https://groups.google.com/forum/#!topic/golang-nuts/yNis7bQG_rY]].)

A Go compiler that was less smart about escape analysis wouldn't
have this problem; as you can see, the compiler has to reason through
several layers of code to go wrong. But escape analysis is an
important optimization for a language like Go so the compiler has
clearly worked hard at it.

(If the Go compiler is doing not just cross function but cross
package escape analysis (which it certainly looks like), I have to
say that I'm impressed by how thorough it's being.)

=== Sidebar: How _escapes()_ works

Before I looked at it, I expected _escapes()_ to involve some deep
magic to tell the compiler to go away. [[The reality
https://github.com/golang/go/blob/4c2465d47d8c706832bbc57668680a3ffc4d800f/src/reflect/value.go#L2499]]
is more prosaic (and humbling for me in my flights of imagination):

 var dummy struct {
     b bool
     x interface{}
 }

 func escapes(x interface{}) {
     if dummy.b {
        dummy.x = x
     }
 }

In theory a sufficiently smart compiler could detect that _dummy_
is not exported from _reflect_ and is not touched inside it, so
_dummy.b_ is always _false_ and _escapes()_ always does nothing and
so _x_ does not escape it.  In practice I suspect that the Go
compiler will never get that perversely smart for various reasons.