== Turning, well copying blobs of memory into Go structures

As before, suppose ([[not entirely hypothetically
https://github.com/siebenmann/go-kstat/]]) that you're writing a
package to connect Go up to something that will provide it with
blobs of memory that are actually C structs; these might be mmap()'d
files, information from a library, or whatever. Once you have [[a
compatible Go _struct_ GoCGoCompatibleStructs]], you still have to
get the data from a C struct (or raw memory) to the Go struct.

One way to do this is to manually write your own struct copy function
that does it field by field (eg '((io.Field = ks_io.field))' for
each field). As with defining the Go structs by hand, this is tedious
and potentially error prone. You can do it and you'll probably have
to if the C struct contains unions or other hard to deal with things,
but we'd like an easier approach. Fortunately there are two good
ones for two different cases. In both cases we will wind up copying
the C struct or the raw memory to a Go struct variable that is [[an
exact equivalent of the C struct GoCGoCompatibleStructs]] (or at
least we hope it is).

The easy case is when we're dealing with a fixed _struct_ that we
have a known Go type for. Assuming that we have a C _void *_ pointer
to the original memory area called ((ks.ks_data)), we can adopt the
C programmer approach and write:

.pn prewrap on

>  var io IO
>  io = *((*IO)(ks.ks_data))
>  return &io

This casts ((ks.ks_data)) to a pointer to an _IO_ struct and then
dereferences it to copy the struct itself into the Go variable we
made for this. Depending on the C type of ((ks_data)), you may need
to use the hammer of _unsafe.Pointer()_ here:

>  io = *((*IO)(unsafe.Pointer(ks.ks_data)))

At this point, some people will be tempted to skip the copying and
just return the 'casted-to-*IO' ((ks.ks_data)) pointer. You don't
want to do this, because ~~if you return a Go pointer to C data,
you're coupling Go and C memory management lifetimes~~. The C
memory must not be freed or reused for something else for as long
as Go retains at least one pointer to it, and there is no way for
you to find out when the last Go reference goes away so that you
can free the C memory.  It's much simpler to treat 'C memory' as
completely disjoint from 'Go memory'; any time you want to move
some information across the boundary, you must copy it. With copying
we know we can free ((ks.ks_data)) safely the moment the copy is
done and the Go runtime will handle the lifetime of the _io_ variable
for us.

The more difficult case is when we don't know what _struct_s we're
dealing with; we're providing the access package, but it's the
callers who actually know the _struct_s are. This situation might
come up in a package for accessing kernel stats, where drivers or
other kernel systems can export custom stats structs. Our access
package can provide specific support for known _struct_s, but we
need an escape hatch for when the callers knows that some specific
kernel system is providing a 'struct whatever' and it wants to
retrieve that (probably into [[an identical Go struct created through
_cgo_ GoCGoCompatibleStructs]]).

The C programmer approach to this problem is _memmove()_. You can
write _memmove()_ in Go with sufficiently perverse use of the
_unsafe_ package, but you don't want to. Instead we can use the
_reflect_ package to create a generic version of the specific 'cast
and copy' code we used above. How to do this wasn't obvious to me
until I did a significant amount of flailing around with the package,
so I'm going to go through the logic of what we're doing in detail.

We'll start with our call signature:

>  func (k *KStat) CopyTo(ptri interface{}) error { ... }

CopyTo takes a pointer to a Go struct and copies our C memory in
((ks.ks_data)) into the struct. I'm going to omit the _reflect_-based
code to check _ptri_ to make sure it's actually a pointer to a
suitable struct in the interests of space, but you shouldn't in
real code. Also, there are a whole raft of qualifications you're
going to want to impose on what types of fields that struct can
contain if you want to at least pretend that your package is somewhat
memory safe.

To actually do the copy, we first need to turn this _ptri_ interface
value into a _reflect.Value_ that is the destination struct itself:

>  ptr := reflect.ValueOf(ptri)
>  dst := ptr.Elem()

We now need to cast ((ks.ks_data)) to a Value with the type 'pointer to
dst's type'. This is most easily done by creating a new pointer of the
right type with the address taken from ((ks.ks_data)):

>  src := reflect.NewAt(dst.Type(), unsafe.Pointer(ks.ks_data))

This is the equivalent of '_``src := ((*IO)(ks.ks_data))''_' in the
type-specific version. Reflect.NewAt is there for doing just
this; its purpose is to create pointers for 'type X at address Y',
which is exactly the operation we need.

Having created this pointer, we then dereference it to copy the
data into _dst_:

>  dst.Set(reflect.Indirect(src))

This is the equivalent of '_io = *src_' in the type-specific
version. We're done.

In my testing, this approach is surprisingly robust; it will deal
with even structs that I didn't expect it to (such as ones with
unexported fields). But you probably don't want to count on that;
it's safest to give _CopyTo()_ straightforward structs with only
exported fields.

On the whole I'm both happy and pleasantly surprised by how easy
it turned out to be to use the reflect package here; I expected it
to require a much more involved and bureaucratic process. Getting
to this final form involved a lot of missteps and unnecessarily
complicated approaches, but the final form itself is about as minimal
as I could expect. A lot of this is due to the existence of
_reflect.NewAt()_, but there's also that _Value.Set()_ works fine
even on complex and nested types.

(Note that while you could use the reflect-based version even for
the first, fixed struct type case, my understanding is that the
reflect package has not insignificant overheads. By contrast the
hard coded fixed struct type code is about as minimal and low
overhead as you can get; it should normally compile down to basically
a memory copy.)

=== Sidebar: preserving Go memory safety here

I'm not fully confident that I have this right, but I think that to
preserve memory safety in the face of this memory copying you must
insure that the target _struct_ type does not contain any embedded
pointers, either explicit ones or ones implicitly embedded into types
like maps, chans, interfaces, strings, slices, and so on.  Fixed-size
arrays are safe because in Go those are just fixed size blocks of
memory.

If you copy a C struct containing pointers into a Go struct containing
pointers, what you're doing is the equivalent of directly returning
the 'casted-to-*IO' ((ks.ks_data)) pointer. You've allowed the
creation of a Go object that points to C memory and you now have
the same C and Go memory lifetime issues. And if some of the pointers
are invalid or point to garbage memory, not only is normal Go code
at risk of bad things but it's possible that the Go garbage collector
will wind up trying to dereference them and take a fault.

(This makes it impossible to easily copy certain sorts of C structures
into Go structures. Fortunately such structures rarely appear in
this sort of C API because they often raise awkward memory lifetime
issues even in C.)