Why Go cares about the difference between unsafe.Pointer and uintptr

January 18, 2018

Go has two things that are more or less the representation of an untyped pointer; uintptr and unsafe.Pointer (which, contrary to appearances, is a built-in type). On the surface this is a little bit odd, because unsafe.Pointer and uintptr can be converted back and forth between each other. Why not have only one representation of a pointer? What's the difference between the two?

The superficial difference is that you can do arithmetic on an uintptr but not on an unsafe.Pointer (or any other Go pointer). The important difference is explicitly pointed out by the unsafe package's documentation:

A uintptr is an integer, not a reference. Converting a Pointer to a uintptr creates an integer value with no pointer semantics. Even if a uintptr holds the address of some object, the garbage collector will not update that uintptr's value if the object moves, nor will that uintptr keep the object from being reclaimed.

Although unsafe.Pointers are generic pointers, the Go garbage collector knows that they point to Go objects; in other words, they are real Go pointers. Through internal magic, the garbage collector can and will use them to keep live objects from being reclaimed and to discover further live objects (if the unsafe.Pointer points to an object that has pointers of its own). Due to this, a lot of the restrictions on what you can legally do with unsafe.Pointers boil down to 'at all times, they must point to real Go objects'. If you create an unsafe.Pointer that doesn't, even for a brief period of time, the Go garbage collector may choose that moment to look at it and then crash because it found an invalid Go pointer.

By contrast, a uintptr is merely a number. None of this special garbage collection magic applies to objects 'referred to' by a uintptr because as just a number, a uintptr doesn't refer to anything. In turn this leads to many of the careful limitations on the various ways that you can turn an unsafe.Pointer into a uintptr, manipulate it, and turn it back. The basic requirement is that you do this manipulation in such a way that the compiler and the runtime can shield the temporary non-pointerness of your unsafe.Pointer from the garbage collector so this temporary conversion will be atomic with respect to garbage collection.

(I think that my use of unsafe.Pointer in my entry on copying blobs of memory into Go structures is safe, but I admit I'm now not completely sure. I believe that there is some magic going on with cgo, since it can safely manufacture unsafe.Pointers that point to C memory, not Go memory.)

PS: As of Go 1.8, all Go pointers must always be valid (I believe including unsafe.Pointers), even if a garbage collection isn't running at the time. If you ever have an invalid pointer stored in a variable or a field, your code can crash merely by updating the field to a perfectly valid value, including nil. See for example this educational Go bug report.

(I was going to try to say something about the internal magic that allows the garbage collector to cope with untyped unsafe.Pointer pointers, but I'm convinced I don't understand enough about it to even say what sort of magic it uses.)


Comments on this page:

By lilydjwg at 2018-01-19 01:55:39:

IMO it's easier to think uintptr as an unsigned pointer-sized integer type, like usize in Rust. The name is confusing.

By Fred Akalin at 2018-02-11 23:47:52:

The magic that lets cgo store pointers to C objects in unsafe.Pointer isn't too magical. The go runtime knows where the go heap is, and simply ignores all others. I think the rule is that unsafe.Pointer must be a valid pointer, but not necessarily one to a Go object.

See https://groups.google.com/forum/#!msg/golang-nuts/ls7Eg7Ye9pU/ye1GLs8dBwAJ and https://github.com/golang/go/issues/21897#issuecomment-330338879 for authoritative-ish sources.

Written on 18 January 2018.
« A recent performance surprise with X on my Fedora Linux desktop
I'm one of those people who never log out from their desktop »

Page tools: View Source, View Normal, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Thu Jan 18 20:19:27 2018
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.