Re: next meeting: Monday Feb 10, 10:00 - 12:00 CET

[Hannes Mehnert <hannes@xxxxxxxxxxx>]

Thanks for the discussion,

On 03/02/2025 20:45, Anil Madhavapeddy wrote:
> > On Feb 3, 2025, at 5:30 PM, Samuel Hym <samuel@xxxxxxxxxxx> wrote:
> >
> > > My understanding of the issue was to munmap of a chunk inside an mmapped 
> > > region, which requires a full mmap/munmap implementation (we don't have 
> > > currently). I'm not sure where the code that causes the ABORT lives and maybe 
> > > it can be difficult to trigger the situation?
> > `abort` would be triggered by a `free(x)` where `x` is not the result
> > of a `malloc`.
> > As I didn’t manage to create an example using the compactor that would
> > run into such a situation and I couldn’t find at which point in the
> > code that could be triggered, I reached out to Sadiq. He confirmed
> > that our implementation is correct.

By "correct", if I understand you correctly, it is "this abort will 
never be triggered"?

> > What happens is that, as pools are allocated by the GC, they will
> > probably follow each other in the virtual memory address space; the
> > compactor might decide to release a pool in the middle of that
> > sequence of pools, so we’ll end up with (coarse-grained) fragmentation
> > with our `malloc`/`free` implementation. But it will be valid
> > nevertheless!

I don't quite understand the impact of that paragraph. "it will be valid 
nevertheless" - does this mean, we won't run into the "abort()"?

> My suggestion to Sadiq when we were discussing the compactor a few months ago 
> was to not reinvent the wheel but to use https://github.com/microsoft/mimalloc 
> instead. It's a state of the art malloc implementation with a very easy 
> integration path to be directly embedded into a Mirage runtime. I think it 
> would probably work out better than using a generic libc malloc in Unikraft 
> (and certainly better than newlib's malloc implementation).

"work out better than using a generic libc malloc" - I'm sure this is a 
low-hanging optimization fruit -- at the moment, ocaml-solo5 uses 
dlmalloc (I'm not sure what Unikraft uses, neither what "newlib's malloc 
implementation" refers to).
If I remember, Pierre did some experiments with another kind of malloc, 
mainly to hunt a memory issue for Xen?
Of course, a full-blown jemalloc (https://github.com/jemalloc/jemalloc) 
may as well be beneficial in terms of performance.
To me, the underlying question is about the tradeoff between code size 
and performance -- and performance should be taken into account with a 
concrete (unikernel) example [plus the variance in terms of bytes vs 
Bigarray, and allocation strategies].

> I put together a quick tree last year that just directly vendored mimalloc, and 
> the integration was pretty straightforward. As always, 80% of the effort will 
> be in the build system integration, which I didn't do ;-)

True.



Hannes