[Xen-devel] FYI: userland <-> hypervisor parameter passing

[Hollis Blanchard <hollisb@xxxxxxxxxx>]

This is the changeset I just committed to the PowerPC tree to pass parameters 
between userland and Xen on PPC. If you'll recall, the problem is that the 
userland tools (libxc) pass virtual addresses all the way down to the 
hypervisor. These pointers are worthless when userspace and the hypervisor 
run in separate address spaces.

The kernel could easily perform the virtual->machine address translation using 
the pfn2mfn arrays, except that these data structures contain virtual 
pointers to still *other* structures. It is not feasible for the kernel to 
understand and munge all the data structures passed in to perform this 
translation, then translate back on the return path.

Another possibility I have not investigated would be to replace the nested 
pointers with nested structures. This would be a more invasive change, and 
currently we're trying to get by on PPC without any/many architecture-neutral 
changes.

From previous conversations, I understand that the x86-64 hypervisor runs in a 
separate virtual address space. However, due to the layout of the x86 page 
tables, it's relatively straightforward (though not ideal) to walk the page 
tables and perform the translation in software. The approach taken in this 
patch may speed up the translation currently done on x86-64, or it may not.

On PowerPC, Xen runs in real mode (untranslated mode), which is also a 
separate address space. However, PowerPC's page tables are not easily 
walkable in software, so that option is not available to us.

Instead, this patch has userspace allocate all structures from the same page, 
with some translation information stuffed at the base of the page. Userland 
records the virtual base of the page, and the kernel records the physical 
base. (We have not yet implemented the pfn2mfn macros, but the hypervisor can 
easily convert physical->machine addresses.) The Xen copy_to/from_user() 
functions operate on virtual addresses, masking with PAGE_SIZE to find the 
needed translation information. This translation is currently only done for 
dom0_op and memory_op hcalls, which are the offenders I've run into so far.

Multi-page data structures are not yet supported. I think we will need to 
allocate the needed pages in userspace, then have kernel and hypervisor 
cooperation to rearrange them to be machine-contiguous. I'm not really 
looking forward to that.

The x86 implementation should be much simpler than PowerPC. xencomm_alloc() 
would be malloc+mlock, and xencomm_free() would be munlock. I have not yet 
implemented this; if there is interest in this approach I may.

[Side note: "domctrl" is a custom domain builder we're using because libxc 
needs some portability love that nobody seems interested in at the moment 
(probably justifiably so). domctrl directly shares the bottom parts of libxc 
anyways, which is why you see libxc patches in this changeset.]

PowerPC Xen tree: http://xenbits.xensource.com/ext/xenppc-unstable.hg
PowerPC Xen Linux tree: http://xenbits.xensource.com/ext/linux-ppc-2.6.hg

-- 
Hollis Blanchard
IBM Linux Technology Center

Attachment: xencomm.diff
Description: Text Data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel