Re: [Xen-devel] [RFC] Generating Go bindings for libxl

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 9/11/19 9:25 PM, Nicholas Rosbrook wrote:
> Hi George,
> 
> I made some more progress on gengotypes.py [1].
[snip]
> What are your thoughts on these implementations so far?

Great!  Overall it looks like it's really making progress, which is
exciting.

> First, I implemented the KeyedUnion translation that we talked about. 
> You can see an example of the generated code in [2].

That looks about like we expected -- tolerable and functional, to be
certain, but LotsOfReallyLongTypeNames.

I was chatting with Anthony today, and thought I'd just toss the idea
out there for discussion.

The only purpose of unions in these structures is to save space (as
opposed to other kinds of unions are specifically designed to allow
different "views" of the same underlying data).  We're replacing the
unions with structures which will be 1) allocated separately, and 2)
require casting and type assertions to handle properly.  This will save
*some* space, but at the cost of a certain amount of complexity, and
run-time overhead.

What we just defined three separate elements in the struct?  E.g.:

type DomainInfo struct {
        /* etc */
        Type struct {
                Key int
                Hvm struct {
                        Firmware string
                        /* ... */
                }
                Pv struct {
                        /* ... */
                }
                Pvh struct {
                        /* ... */
                }
        }
}

This obviously means keeping a whole load of useless HVM and PV fields
around when you just want to run PVH, but it you can simply do this:

if ( di.Type.Key == libxl.DomainTypeHvm ) {
   /* ... */
   firmware := di.Type.Hvm.Firmware;
}

This also mean you could make a mistake and access the HVM fields for a
PV domain, and you'd get neither a compile-time nor a run-time error.

Anyway, like I said, just tossing it out there.  If we decide we don't
want duplicate structs, I think your implementation looks about as good
as it can be.

> Second, I took a first pass at the C-to-Go type marshaling. I defined a
> "marshaler" interface in [3], which allows the convenience function
> `func fromC(m marshaler, ctype interface{}) (err error)`. My primary 
> motivation for this interface is to allow the generated code to call panic
> rather than checking for and handling errors. However, the previously
> mentioned convenience function will recover from those panics and return
> the appropriate error. So, each generated struct implements this interface.
> See the generated code in [4].

So the advantage of this is that you can just call:

    fromC(&di, &cdi)

Rather than:

    di.fromC(&cdi)

?

But the cost for this is that we're switching from static type-checking
to dynamic type-checking.  If in the first example, cdi is the wrong
type (for instance, if we forget the & at the front), everything will
compile, and we won't notice unless the function actually gets called.
In the second example, if we're not trying to implement a generic
"marshaler" method, we can define the function signature to specify
exactly what pointer we need.

I don't want to say I'd rule it out, but it doesn't seem to me like the
convenience is worth the cost (unless there's another advantage I'm
missing).

> You'll also notice in [4] that I defined C structs in the cgo preamble which
> correspond to the Go KeyedUnion structs, e.g. DomainBuildInfoTypeUnionPv.
> Since cgo treats C unions a byte slice, we need to do an unsafe.Pointer 
> conversion
> to some struct to be able to access the fields of a union. So, I thought it 
> would
> make the most sense to do the cast to a C type, and then convert those fields
> to Go types accordingly. See [5] for example.

Right -- that looks like just about the only option?  Anyway, it's a
good option; no point spending a lot of time looking for ways to improve
something that's only really going to live inside one generated file.

> I was able to write a couple examples to demonstrate the generated code is 
> working, but I had to make some small changes to the existing code WRT
> libxl builtin types (not committed to my branch). So, I thought we should 
> decide
> how these builtin types will be defined in Go. This is what I was thinking so 
> far:
> 
> Defbool (?)

Well this is really defined by the interface.  libxl.h has this defined as

struct {
   int val;
}


We could basically do the same thing; but make `val` non-exported.

> Domid (already exists)
> Devid => int
> Uuid => [16]byte
> Mac => [6]byte
> Bitmap (already exists)

Yup

> CpuidPolicyList (?)

So this is an interesting one.  libxl__cpuid_policy is essentially a
type containing all non-exported fields.  (i.e., the actual elements are
defined inside libxl_internal.h, and the outside world only gets pointers).

And it's a list terminated by a specific value inside
libxl__cpuid_policy, which means the list itself is basically opaque
entirely.

I'd be tempted to say just do something like:

type CpuidPolicyList struct {
    val C.libxl_cpuid_policy_list
};

A part of me thinks even something like this wouldn't be terrible:

type CpuidPolicyList C.libxl_cpuid_policy_list

It "leaks" the internals out to the callers, but it also means you don't
have to do all this faff of marshalling / unmarshalling what's
essentially just a pointer.

> StringList => [ ]string
> KeyValueList => map[string]string
> Hwcap (already exists, but should be re-factored to be like Bitmap to hide 
> the C type)
> MsVmGenid => [16]byte

Should probably be C.LIBXL_MS_VM_GENID_LEN, but yes.

> EvLink (?)

It sort of looks like this is an entirely internal thing that libxl
uses.  I think to begin with we can just declare this as an empty
struct, and figure out what to put in it once it becomes more clear how
it needs to be used.

Thanks for all your work on this!

Peace,
 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel