Re: [Xen-devel] Re: how to handle paged hypercall args?

[Keir Fraser <keir@xxxxxxx>]

On 15/11/2010 12:04, "Tim Deegan" <Tim.Deegan@xxxxxxxxxx> wrote:

> At 11:55 +0000 on 15 Nov (1289822118), Keir Fraser wrote:
>> The issue is that there are hundreds of uses of the guest-accessor macros.
>> Every single one would need updating to handle the paged-out-so-retry case,
>> unless we can hide that *inside* the accessor macros themselves. It's a huge
>> job, not to mention the bug tail on rarely-executed error paths.
> 
> Right, I see.  You're suggesting that we code up a sort of setjmp() that
> can be called in the __copy function, which will deschedule the vcpu and
> allow it to be rescheduled back where it was.  Sounds ideal.

Exactly so.

> Will it
> need per-vcpu stacks? (and will they, in turn, use order>0 allocations? :))

Of a sort. I propose to keep the per-pcpu stacks and then copy context
to/from a per-vcpu memory area for the setjmp-like behaviour. Guest call
stacks won't be very deep -- I reckon a 1kB or 2kB per-vcpu area will
suffice.

In some ways this is a backwards version of the Linux stack-handling logic,
which has a proper per-task kernel stack which is of moderate size (4kB?).
Then it has per-cpu irq stacks which are larger to deal with deep irq
nesting. We will have proper per-cpu hypervisor stacks of sufficient size to
deal with guest and irq state -- our per-vcpu 'shadow stack' will then be
the special case and only of small/moderate size to deal with shallow guest
call stacks.

> We'll have to audit the __copy functions to make sure they're not called
> with locks held.  Sounds more fun than the alternative, I guess.

Exactly so. Best of a bad set of options. At least we can run-time assert
this and it's not error-path only.

> I think the ioreq code would be another candidate for tidying up if we
> had such a mechanism.  Presumably some of the current users of
> hypercall_create_continuation() would benefit too.

Yeah, it needs a dash of thought but I think we will be able to move in this
direction.

 -- Keir

>> Consider also the copy_to_* writeback case at the end of a hypercall. You've
>> done the potentially non-idempotent work, you have some state cached in
>> hypervisor regs/stack/heap and want to push it out to guest memory. The
>> guest target memory is paged out. How do you encode the continuation for the
>> dozens of cases like this without tearing your hair out?
>> 
>> I suppose *maybe* you could check-and-pin all memory that might be accessed
>> before the meat of a hypercall begins. That seems a fragile pain in the neck
>> too however.
> 
> Good point.
> 
> Tim.



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel