Getting C-compatible structs in Go with and for cgo

August 30, 2015

Suppose, not entirely hypothetically, that you're writing a package to connect Go up to something that will provide it blobs of memory that are C structs. These structs might be the results of making system calls or they might be just informational things that a library provides you. In either case you'd like to pass these structs on to users of your package so they can do things with them. Within your package you can use the cgo provided C.<whatever> types directly. But this is a bit annoying (they don't have native Go types for things like integers, which makes interacting with regular Go code a mess of casts) and it doesn't help other code that imports your package. So you need native Go structs, somehow.

One way is to manually define your own Go version of the C struct. This has two drawbacks; it's tedious (and potentially error-prone), and it doesn't guarantee that you'll wind up with exactly the same memory layout that C has (the latter is often but not always important). Fortunately there is a better approach, and that is to use cgo's -godefs functionality to more or less automatically generate struct declarations for you. The result isn't always perfect but it will probably get you most of the way.

The starting point for -godefs is a cgo Go source file that declares some Go types as being some C types. For example:

// +build ignore
package kstat
// #include <kstat.h>
import "C"

type IO C.kstat_io_t
type Sysinfo C.sysinfo_t

const Sizeof_IO = C.sizeof_kstat_io_t
const Sizeof_SI = C.sizeof_sysinfo_t

(The consts are useful for paranoid people so you can later cross-check the unsafe.Sizeof() of your Go types against the size of the C types.)

If you run 'go tool cgo -godefs <file>.go', it will print out to standard output a bunch of standard Go type definitions with exported fields and everything. You can then save this into a file and use it. If you think the C types may change, you should leave the generated file alone so you won't have a bunch of pain if you have to regenerate it; if the C types are basically fixed, you can annotate the generated output with eg godoc comments. Cgo worries about matching types and it will also insert padding where it existed in the original C struct.

(I don't know what it does if the original C struct is impossible to reconstruct in Go, for instance if Go requires padding where C doesn't. Hopefully it complains. This hope is one reason you may want to check those sizeofs afterwards.)

The big -godefs limitation is the same limitation as cgo has in general: it has no real support for C unions, since Go doesn't have them. If your C struct has unions, you're on your own to figure out how to deal with them; I believe cgo translates them as appropriate sized uint8 arrays, which is not too useful to actually access the contents.

There are two wrinkles here. Suppose you have one struct type that embeds another struct type:

struct cpu_stat {
   struct cpu_sysinfo  cpu_sysinfo;
   struct cpu_syswait  cpu_syswait;
   struct vminfo       cpu_vminfo;
}

Here you have to give cgo some help, by creating Go level versions of the embedded struct types before the main struct type:

type Sysinfo C.struct_cpu_sysinfo
type Syswait C.struct_cpu_syswait
type Vminfo  C.struct_cpu_vminfo

type CpuStat C.struct_cpu_stat

Cgo will then be able to generate a proper Go struct with embedded Go structs in CpuStat. If you don't do this, you get a CpuStat struct type that has incomplete type information; the 'Sysinfo' et al fields in it will refer to types called _Ctype_... that aren't defined anywhere.

(By the way, I do mean 'Sysinfo' here, not 'Cpu_sysinfo'. Cgo is smart enough to take that sort of commonly seen prefix off of struct field names. I don't know what its algorithm is for doing this, but it's at least useful.)

The second wrinkle is embedded anonymous structs:

struct mntinfo_kstat {
   ....
   struct {
      uint32_t srtt;
      uint32_t deviate;
   } m_timers[4];
   ....
}

Unfortunately cgo can't deal with these at all. This is issue 5253, and you have two options. The first is that at the moment, the proposed CL fix still applies to src/cmd/cgo/gcc.go and works (for me). If you don't want to build your own Go toolchain (or if the CL no longer applies and works), the other solution is to create a new C header file that has a variant of the overall struct that de-anonymizes the embedded struct by creating a named type for it:

struct m_timer {
   uint32_t srtt;
   uint32_t deviate;
}

struct mntinfo_kstat_cgo {
   ....
   struct m_timer m_timers[4];
   ....
}

Then in your Go file:

...
// #include "myhacked.h"
...

type MTimer C.struct_m_timer
type Mntinfo C.struct_mntinfo_kstat_cgo

Unless you made a mistake, the two C structs should have the same sizes and layouts and thus be totally compatible with each other. Now you can use -godefs on your version, remembering to make an explicit Go type for m_timer due to the first wrinkle. If you feel bold (and you don't think you'll need to regenerate things), you can then reverse this process in the generated Go file, re-anonymizing the MTimer type into the overall struct (since Go supports that perfectly well). Since you're not changing the actual contents, just where types are declared, the result should be layout-identical to the original.

PS: the file that's input to -godefs is set to not be built by the normal 'go build' process because it is only used for this godefs generation. If it gets included in the build, you'll get complaints about multiple definitions of your (Go) types. The corollary to this is that you don't need to have this file and any supporting .h files in the same directory as your regular .go files for the package. You can put them in a subdirectory, or keep them somewhere entirely separate.

(I think the only thing the package line does in the godefs .go file is set the package name that cgo will print in the output.)

Written on 30 August 2015.
« The mailing list thread to bug tracking system problem
Turning, well copying blobs of memory into Go structures »

Page tools: View Source, Add Comment.
Search:
Login: Password:
Atom Syndication: Recent Comments.

Last modified: Sun Aug 30 03:37:54 2015
This dinky wiki is brought to you by the Insane Hackers Guild, Python sub-branch.