Here is something that I didn't fully grasp and understand until I did some digging in the Go compiler as part of writing a recent entry:

Despite being in a package, unsafe.Pointer is really a built in type in the Go compiler, just like map, chan, string, and so on.

While there is an unsafe package and it looks somewhat superficially like reflect (which is a real package with real code), this is an illusion. All parts of unsafe are implemented at compile time inside the compiler and as part of this unsafe.Pointer is a built in type, much like uintptr (or more to the point, something complex like chan, map, or slices). One consequence of this is that nothing involving an unsafe.Pointer (such as how it interacts with escape analysis) can be understood or predicted by thinking about it at the Go level through analogies to regular pointers or the like. unsafe.Pointer is not just a little bit magical; it's a lot magical.

(See eg TUNSAFEPTR in go.go. The unsafe functions are interpreted at compile time in unsafe.go, which turns all of them into literals of type uintptr.)

I imagine a large reason that unsafe.Pointer is not simply Pointer is so that people are forced to explicitly import unsafe in order to use it. This probably both avoids some casual use and makes it easier to find (or check for) potential dangerous things; all you have to do is scan things looking for imports of unsafe.

(There's also that 'go tool compile' accepts a -u argument, which disables use of unsafe as part of its effects.)

(Perhaps this has been obvious to other people, but it wasn't to me, especially given that reflect seems to involve at most a tiny little bit of special compiler magic; in Go 1.5, it's a lot of real Go code in a real package.)