[Xen-devel] Proposal to allow setting up shared memory areas between VMs from xl config file

[Zhongze Liu <blackskygg@xxxxxxxxx>]

Hi, Xen developers,

I'm Zhongze Liu, a GSoC student of this year. Glad to meet you in the
Xen Project.  As an initial step to implementing my GSoC proposal, which
is still a draft,  I'm posting it here. And hope to hear from you your
suggestions.

====================================================
1. Motivation and Description
====================================================
Virtual machines use grant table hypercalls to setup a share page for
inter-VMs communications. These hypercalls are used by all PV
protocols today. However, very simple guests, such as baremetal
applications, might not have the infrastructure to handle the grant table.
This project is about setting up several shared memory areas for inter-VMs
communications directly from the VM config file.
So that the guest kernel doesn't have to have grant table support to be
able to communicate with other guests.

====================================================
2. Implementation Plan:
====================================================

======================================
2.1 Introduce a new VM config option in xl:
======================================
The shared areas should be shareable among several VMs,
every shared physical memory area is assigned to a set of VMs.
Therefore, a “token” or “identifier” should be used here to uniquely
identify a backing memory area.


I would suggest using an unsigned integer to serve as the identifier.
For example:

In xl config file of vm1:

    static_shared_mem = [“addr_range1= ID1”, “addr_range2 = ID2”]

In xl config file of vm2:

    static_shared_mem = [“addr_range3 = ID1”]

In xl config file of vm3:

    static_shared_mem = [“addr_range4 = ID2”]


In the example above. A memory area A1 will be shared between
vm1 and vm2 -- vm1 can access this area using addr_range1
and vm2 using addr_range3. Likewise, a memory area A2 will be
shared between vm1 and vm3 -- vm1 can access A2 using addr_range2
and vm3 using addr_range4.

The shared memory area denoted by an identifier IDx will be
allocated when it first appear, and the memory pages will be taken from
the first VM whose static_shared_mem list contains IDx. Take the above
config files for example, if we instantiate vm1, vm2 and vm3, one after
another, the memory areas denoted by ID1 and ID2 will both be allocated
in and taken from vm1.

======================================
2.2 Store the mem-sharing information in xenstore
======================================
This information should include the length and owner of the area. And
it should also include information about where the backing memory areas
are mapped in every VM that are using it. This information should be
known to the xl command and all domains, so we utilize xenstore to keep
this information. A current plan is to place the information under
/local/shared_mem/ID. Still take the above config files as an example:

If we instantiate vm1, vm2 and vm3, one after another,
“xenstore ls -f” should output something like this:


After VM1 was instantiated, the output of “xenstore ls -f”
will be something like this:

    /local/shared_mem/ID1/owner = dom_id_of_vm1

    /local/shared_mem/ID1/size = sizeof_addr_range1

    /local/shared_mem/ID1/mappings/dom_id_of_vm1 = addr_range1


    /local/shared_mem/ID2/owner = dom_id_of_vm1

    /local/shared_mem/ID2/size = sizeof_addr_range1

    /local/shared_mem/ID2/mappings/dom_id_of_vm1 = addr_range2


After VM2 was instantiated, the following new lines will appear:

    /local/shared_mem/ID1/mappings/dom_id_of_vm2 = addr_range3


After VM2 was instantiated, the following new lines will appear:

    /local/shared_mem/ID2/mappings/dom_id_of_vm2 = addr_range4

When we encounter an id IDx during "xl create":

  + If it’s not under /local/shared_mem, create the corresponding entries
     (owner, size, and mappings) in xenstore, and allocate the memory from
     the newly created domain.

  + If it’s found under /local/shared_mem, map the pages to the newly
      created domain, and add the current domain to
      /local/shared_mem/IDx/mappings.

======================================
2.3 mapping the memory areas
======================================
Handle the newly added config option in tools/{xl, libxl}
and utilize toos/libxc to do the actual memory mapping


======================================
2.4 error handling
======================================
Add code to handle various errors: Invalid address,
mismatched length of memory area etc.

====================================================
3. Expected Outcomes/Goals:
====================================================
A new VM config option in xl will be introduced, allowing users to setup
several shared memory areas for inter-VMs communications.
This should work on both x86 and ARM.

[See also: 
https://wiki.xenproject.org/wiki/Outreach_Program_Projects#Share_a_page_in_memory_from_the_VM_config_file]


Cheers,

Zhongze Liu

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel