Re: [Xen-devel] [PATCH v6 08/12] libxl: ocaml: drop the ocaml heap lock before calling into libxl

[Rob Hoes <Rob.Hoes@xxxxxxxxxx>]

On 29 Nov 2013, at 10:48, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Fri, 2013-11-29 at 09:03 +0000, Rob Hoes wrote:
>> On 29 Nov 2013, at 08:39, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
>>> I wonder if you need to use these functions:
>>>> /* [caml_register_global_root] registers a global C variable as a memory 
>>>> root
>>>>  for the duration of the program, or until [caml_remove_global_root] is
>>>>  called. */
>>>> 
>>>> CAMLextern void caml_register_global_root (value *);
>>>> 
>>>> /* [caml_remove_global_root] removes a memory root registered on a global C
>>>>  variable with [caml_register_global_root]. */
>>>> 
>>>> CAMLextern void caml_remove_global_root (value *);
>>> 
>>> To keep the for_callback value live?
>> 
>> Perhaps, yes, if a global root is something that is not only never
>> deallocated, but also never moved. I remember we discussed this topic
>> when revising v1 of the series, and I ended up with the current
>> implementation that keeps a reference to the value at the ocaml level.
>> 
>> I wonder how ocaml tracks live values when they may move around. I’ll
>> see what I can find out.
> 
> My google-fu wasn't sufficient to answer this, perhaps you have a
> hotline to someone who knows though ;-)

Right, I found out some more about this.

Ocaml’s GC works on two generations. There is a small minor heap for the newest 
values, and a major heap for the older ones. When the minor heap gets full, the 
GC moves all values that can be reached from the set of GC roots to the major 
heap using a "Stop&Copy" algorithm, and then considers the whole minor heap as 
free space. Space from the major heap is reclaimed at a slower pace using a 
“Mark&Sweep” algorithm, which basically frees up unused values in place. 
Occasionally, the major heap is compacted (defragmented). See 
http://caml.inria.fr/pub/docs/oreilly-book/html/book-ora087.html (and the page 
before it) for details.

All this means that it is quite likely indeed that a value is moved around at 
least once if it stays around for a bit.

This of course has implications for C bindings. It turns out that the 
CAMLparam/CAMLlocal/CAMLreturn macros don’t just protect values from being 
destroyed by the GC. They also allow the GC to update the pointers when values 
move around (the “value” type in C is a pointer to something on the ocaml 
heap). (There is an interesting thread that talks about this, here: 
http://groups.yahoo.com/neo/groups/ocaml_beginners/conversations/topics/11278).

This first of all explains why those macros need to be used in all functions 
that use ocaml values, even if they are not primitives that are directly called 
by ocaml, such as aohow_val. Only if the function does absolutely no 
allocations, and the heap lock is not dropped, it may be safe to omit them.

Secondly, storing a value (ocaml heap pointer) in C for a while, and expecting 
it to still be up-to-date after a while, is wrong. In the case of the 
for_callback pointer in libxl, we currently give it the our ocaml value 
directly, and therefore may get out-of-date. I think that one way to fix this, 
is to store the value in a global C variable which is registered with the GC 
using caml_register_global_root, and use a pointer to _that_ as for_callback. 
Or we could use some other C-allocated variables to pass around, and maintain a 
mapping to the ocaml values.

Actually, there is one case for which the current code does actually seem 
alright: when using integers. In this case, the “value” is not a pointer, but 
the integer itself.

Cheers,
Rob
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel