Re: [Xen-devel] [RFC] Extending numbers of event channels

[Wei Liu <Wei.Liu2@xxxxxxxxxx>]

On Mon, 2012-12-03 at 17:35 +0000, Jan Beulich wrote:
> >>> On 03.12.12 at 17:29, Wei Liu <Wei.Liu2@xxxxxxxxxx> wrote:
> > Regarding Jan's comment in [0], I don't think allowing user to specify
> > arbitrary number of levels a good idea. Because only the last level
> > should be shared among vcpus, other level should be in percpu struct to
> > allow for quicker lookup. The idea to let user specify levels will be
> > too complicated in implementation and blow up percpu section (since the
> > size grows exponentially). Three levels should be quite enough. See
> > maths below.
> 
> I didn't ask to implement more than three levels, I just asked for
> the interface to establish the number of levels a guest wants to
> use to allow for higher numbers (passing of which would result in
> -EINVAL in your implementation).
> 

Ah, I understand now. How about something like this:

struct EVTCHNOP_reg_nlevel {
    int levels;
    void *level_specified_reg_struct;
}

> > Number of event channels:
> >  * 32bit: 1024 * sizeof(unsigned long long) * BITS_PER_BYTE = 64k
> >  * 64bit: 4096 * sizeof(unsigned long long) * BITS_PER_BYTE = 512k
> > Basically the third level is a new ABI, so I choose to use unsigned long
> > long here to get more event channels.
> 
> Please don't: This would make things less consistent to handle
> at least in the guest side code. And I don't see why you would
> have a need to do so anyway (or else your argument above
> against further levels would become questionable).
> 

It was suggested by Ian to use unsigned long long. Ian, why do you
prefer unsigned long long to unsigned long?

> > Pages occupied by the third level (if PAGE_SIZE=4k):
> >  * 32bit: 64k  / 8 / 4k = 2
> >  * 64bit: 512k / 8 / 4k = 16
> > 
> > Making second level percpu will incur overhead. In fact we move the
> > array in shared info into percpu struct:
> >  * 32bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 128 byte
> >  * 64bit: sizeof(unsigned long) * 8 * sizeof(unsigned long) = 512 byte
> > 
> > What concerns me is that the struct evtchn buckets are allocated all at
> > once during initialization phrase. To save memory inside Xen, the
> > internal allocation/free scheme for evtchn needs to be modified. Ian
> > suggested we do small number of buckets at start of day then dynamically
> > fault in more as required.
> > 
> > To sum up:
> >      1. Guest should allocate pages for third level evtchn.
> >      2. Guest should register third level pages via a new hypercall op.
> 
> Doesn't the guest also need to set up space for the 2nd level?
> 

Yes. That will be embedded in percpu struct vcpu_info, which will be
also register via the same hypercall op.


Wei.

> Jan
> 
> >      3. Hypervisor should setup third level evtchn in that hypercall op.
> >      4. Only last level (third in this case) should be shared among
> >         vcpus.
> >      5. Need a flexible allocation/free scheme of struct evtchn.
> >      6. Debug dumping should use snapshot to avoid holding event lock
> >         for too long. (Jan's concern in [0])
> > 
> > Any comments are welcomed.
> > 
> > 
> > Wei.
> > 
> > [0] http://thread.gmane.org/gmane.comp.emulators.xen.devel/139921 
> 
> 
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel