RE: [Xen-devel] Grant tables from dom0 userspace?

["King, Steven R" <steven.r.king@xxxxxxxxx>]

Hi Andrew -- 

> Are there implicit unmapping problems that you thing would remain
unsolved if,
> for instance, there was a vm area destructor early enough to allow the
OS to
> properly unmap active grants?

I'm having a post-unmap balloon driver problem, but could be my fault
(see below).  I'm  not convinced that implicit unmapping is an
unreasonable thing to ask of Xen.  It feels less complex overall to have
Xen deal with this once, rather than N operating system deal with it N
different ways.  I'd be interested to hear arguments here.

> When you say, "others remain," can you provide a little more detail?
I tried to list some in my reply to Keir.

While I've got you on the line... :^)  I'm having two problems right now
with my attempt at implicit unmap.  The hacks let me limp along well
enough to make progress on the code that uses the shared memory, but any
thoughts on further improvements are most welcome.

The implicit unmap code executes when the cleanup_writable_pagetable()
code _PAGE_GNTMAP flag is found in a modified pte.  The backtrace looks
like this:
(XEN)    [<ff13603e>] put_page_from_l1e+0xd0/0x1af
(XEN)    [<ff13a891>] revalidate_l1+0x159/0x168
(XEN)    [<ff13aac1>] ptwr_flush+0x221/0x32f
(XEN)    [<ff13b6a7>] cleanup_writable_pagetable+0x5c/0x7d
(XEN)    [<ff137c20>] do_mmuext_op+0x85/0x8c1
(XEN)    [<ff149e0f>] hypercall+0x8f/0xaf

At this point, the Linux has just squashed the pte.  Xen code knows the
l1e, and I've added a few more bits to the maptrack object to allow me
to find a match from the corresponding pfn.  It's kludgey, because this
only works in the special case where only one map track entity exists
for a given pfn per domain.  This is problem #1.  Ideas?  If we had the
exact address of the pte, there would be no ambiguity in which maptrack
entry owns the mapping.  I nosed around and did not find a way to get
the pte addr, but still hold out hope that it's possible.

Armed with the "correct" maptrack entry, I can do most of the ummapping
work then and there.  Later, I get the "late" vma_close() from the OS
where my guest driver explicitly unmaps to finish the job.

All is well, and I'm back at the guest command prompt.  Problem #2 is
that the balloon driver crashes me if I try to restart my user-level
app.  The dump looks like this:

kernel BUG at drivers/xen/balloon/balloon.c:218!
invalid operand: 0000 [#1]
SMP
Modules linked in:
CPU:    0
EIP:    0061:[<c023d8c4>]    Not tainted VLI
EFLAGS: 00010097   (2.6.14-xenU)
EIP is at increase_reservation+0x1c4/0x230
eax: c1000000   ebx: 0000f377   ecx: c11e6eec   edx: c0042000
esi: 00000004   edi: c11e6ee0   ebp: c038bf18   esp: c038bed8
ds: 007b   es: 007b   ss: 0069
Process events/0 (pid: 5, threadinfo=c038a000 task=c03b3530)
Stack: c11e6e80 c0346d00 00000000 00000000 cf3ac000 cf3ac000 00000004
00000000
       00000000 00007ff0 cf8f5030 c03b3658 00000005 00000004 00000000
c038a000
       c038bf34 c023db8a 00000004 c0132dff 00000000 c03b1c00 c02ff004
c038bfb4
Call Trace:
 [<c010845f>] show_stack+0x7f/0xa0
 [<c0108612>] show_registers+0x162/0x1d0
 [<c0108824>] die+0xf4/0x180
 [<c02adb85>] do_trap+0xb5/0xc0
 [<c0108bec>] do_invalid_op+0xbc/0xd0
 [<c0108127>] error_code+0x2b/0x30
 [<c023db8a>] balloon_process+0x3a/0xb0
 [<c012e05c>] worker_thread+0x19c/0x240
 [<c0132ada>] kthread+0xba/0xc0
 [<c0105f49>] kernel_thread_helper+0x5/0xc
Code: 00 00 00 00 31 d2 89 f8 e8 5a 51 f0 ff ff 45 cc 8b 55 08 39 55 cc
0f 82 6b ff ff ff e9 2b ff ff ff 0f 0b e7 00 3a 92 2d c0 eb cd <0f> 0b
da 00 3a 92 2d c0 e9 76 ff ff ff 0f 0b d7 00 3a 92 2d c0

I haven't pursed this one at all, so it could be the result a dumb
mistake on my part.

-steve


-----Original Message-----
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Andrew
Warfield
Sent: Thursday, March 09, 2006 10:59 AM
To: King, Steven R
Cc: Cihula, Joseph; Jacob Gorm Hansen; xen-devel Devel
Subject: Re: [Xen-devel] Grant tables from dom0 userspace?

Hi Steven,

> I've hacked around many of
> the problems, such as implicit unmapping of grant references, but 
> others remain.  Some of the issues have no resolution in the grant 
> table architecture.

   It would be quite useful to know what specific problems you have
encountered in this effort as insight for future versions of the code.
 The main problem that I have encountered in user-level foreign page
mappings is (as discussed previously) that Linux is a bit hasty in
blowing away user virtual memory areas, and doesn't provide a good
mechanism to safely unmap the pages.  The kernel has all the information
it needs to do the right thing though, and I think this should be
reasonably fixed in the future -- i don't see a compelling reason for
Xen to do extra book-keeping to cover for something that the OS could
just as easily do.  Especially for something like cross-VM mappings,
which don't exist in the non-virtualized system. 
Are there implicit unmapping problems that you thing would remain
unsolved if, for instance, there was a vm area destructor early enough
to allow the OS to properly unmap active grants?

   When you say, "others remain," can you provide a little more detail?

thanks!
a.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel