Re: [Xen-devel] Shared memory between a process in dom0 and a process in domU

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Wed, 2013-07-24 at 14:46 +0000, SÃbastien FREMAL [530784] wrote:
> Hello,
> 
> I'm trying to share memory pages between a process executing in dom0
> and a process executing in a domU. I work with xen 4.3 and linux 3.2
> (it's necessary due to a compatibility issue). I succeeded in
> accessing pages from a process in dom0, but I'm encountering problems
> to access them from a domU. I tried two methods :
> 1) I allocate a vm area in which I put the pages identified by their
> grant references. This step works, but I don't find how to retrieve
> the pages in the mmap function to map them into user space. Here is
> the corresponding code (it's based on the xensocket code) :
> 
> if (!(x->buffer_area = alloc_vm_area(buffer_num_pages * PAGE_SIZE,
> NULL))) {
>     DPRINTK("error: cannot allocate %d buffer pages\n",
> buffer_num_pages);
>     goto err_unmap;
> }
> 
> //x->buffer_addr is a pointer used to access more easily to
> x->buffer_area->addr
> x->buffer_addr = (unsigned long)x->buffer_area->addr;
> 
> //we retrieve the first gref
> grefp = &d->buffer_first_gref;
> for (i = 0; i < buffer_num_pages; i++) {
> 
>     memset(&op, 0, sizeof(op));
>     op.host_addr = x->buffer_addr + i * PAGE_SIZE;
>     op.flags = GNTMAP_host_map;
>     op.ref = *grefp;
>     op.dom = x->otherend_id;
> 
>     rc = HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &op, 1);
>     if (rc == -ENOSYS) {
>         goto err_unmap;
>     }
> 
>     if (op.status) {
>            DPRINTK("error: grant table mapping failed\n");
>         goto err_unmap;
>     }
> 
>     x->buffer_handles[i] = op.handle;
>     grefp = (int *)(x->buffer_addr + i * PAGE_SIZE);
> }
> 
> Now, the grefs are mapped and the data they contain is accessible from
> the module. I instantiated a mmap function to access them from a user
> process :
> 
> //I first tried to get the PFN of the page to map it with
> remap_pfn_range, but it doesn't work as vmalloc_to_pfn returns 0 and
> not the PFN
> unsigned long pfn = vmalloc_to_pfn((void *)(x->buffer_addr
> +recv_offset*PAGE_SIZE));
> if(pfn_valid(pfn)){
>     printk(KERN_INFO "PFN is valide\n");
> }
> 
> //I studied how gntdev worked and I realised it uses vm_insert_page, I
> then tried to retrieve the struct page instead of the PFN
> struct page * shPage = vmalloc_to_page((void *)(x->buffer_addr
> +recv_offset*PAGE_SIZE));
> if(shPage == NULL)
>     printk(KERN_INFO "Page not found\n");
> 
> vma->vm_flags |= VM_RESERVED;
> 
> //This function fails with the error code -22 (invalid value)
> int err = vm_insert_page(vma, vma->vm_start, shPage);
> if (err)
>     printk(KERN_INFO "Error while inserting a page\n");
> else
>     printk(KERN_INFO "Page inserted !!");
> 
> I don't understand why this method fails. vmalloc_to_page returns a
> struct page *, but the function page_count (called by vm_insert_page)
> fails when called for this page.

What do you mean by "fails"? That function returns the current value of
the page's reference count it cannot really fail as such.

>  Is there a subtlety specific to Xen to allow this operation to
> succeed ?

Not as far as I know, your issue appears to be a generic Linux one. I
don't know the answer off hand but you might want to try one of the
Linux lists/newgroups perhaps?

> I don't really know what happens here, but when gnttab_map_refs is
> called, the calling domU is rebooted and its ID is changed (from 1 to
> 2 i.e.).

Perhaps the guest tried something so illegal that it was killed and the
toolstack rebooted it? You should see evidence of this in either the
hypervisor logs or the toolstack logs.

> Can someone please tell me what's wrong in my code ? I just have to
> get the right page in the first method to make it work.

If what you want to do is share memory between dom0 and domU userspace
then you should look into the gntdev and gntshr device drivers which
already exist to support this usecase and will mean you don't need to
write any drivers at all.

For an example of the use of these existing drivers you could look at
libvchan (tools/libvchan). In fact if what you want is a shared memory
ring protocol between two processes then libvchan may already do
everything you need.

Ian.




_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel