In Go, unsafe type conversions are still garbage collection safe

May 3, 2019

I was recently reading to slice or not to slice (via), where one of the example ways of going from a slice to an array is the non copying brute force approach with unsafe. To quote the first part of the example code:

bufarrayptr := (*[32]byte)(unsafe.Pointer(&buf[0])) // *[32]byte (same memory region)

(Here buf is a slice, and we'll assume that we've already verified that it has a len() of at least 32.)

One of the things you might wonder about here is whether this is safe from garbage collection. After all, we're eventually discarding buf, the original and true reference to the slice and its backing array. Will the Go garbage collector someday free the memory involved even though we still have a reference to it in the form of bufarrayptr? On the one hand, we do have a reference to the backing array; on the other hand, we created the reference through the use of unsafe and thus went behind Go's back to do it.

(This would be an analog of the C mistake of retaining a pointer to something that you've free()'d.)

Conveniently, the answer is that this unsafe type conversion is still safe from being garbage collected. As I discussed in Exploring how and why interior pointers keep entire objects alive, the Go garbage collector is blind to the type of pointers; it intrinsically knows what a particular block of memory is, and any pointer to it of any type is as good as any other pointer. It does not have to be the original type, and in fact it can generally be a completely incorrect type. As far as garbage collection goes, I suspect that you can get away with a pointer of a type that is larger than what you're actually pointing to.

(The Go language specification does not say that this has to work, though, although it does say a bit about unsafe. The unsafe package itself implicitly says that reinterpreting to a too-large type is invalid usage.)

There is an important consequence of this type blindness in the garbage collector if you are doing type conversions, and that is what is and isn't a pointer is set permanently from the original type. When the Go runtime allocates memory initially, it records the 'shape' of that memory, including what portions of it are pointers. Regardless of how you reinterpret the memory, that original shape sticks to it, and that original shape is what the garbage collector uses when determining what pointers to follow to find more used memory and what bytes to not look at further because they're not pointers. If you put non-pointer data in what the garbage collector thinks is a pointer, it will probably panic your program. If you put pointer data in what the garbage collector thinks is not a pointer, the garbage collector may decide that some memory is unused and free it even though you think you have a pointer to it; when you use that pointer later, you will be sad.

(In general, reinterpreting non-pointer memory as a pointer is not necessarily safe. The Go runtime does some things when you tell it you're modifying a pointer, and it makes no guarantees that those things will be safe if the actual memory did not start out its life as a pointer.)

PS: I think that this garbage collection safe behavior of unsafe.Pointer is implicitly guaranteed by the first unsafe.Pointer usage pattern. This isn't part of the language specification itself but it is part of the current unsafe package specification, so it's pretty close. As a practical matter, I think that the Go authors see this sort of usage as valid and thus are likely to support it for as long as possible.

Written on 03 May 2019.
« Some implications of using offset instead of delta() in Prometheus
What usually identifies an intermediate or root TLS certificate »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Fri May 3 22:17:16 2019
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.