Re: [Xen-devel] Xen on ARM IRQ latency and scheduler overhead

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Fri, 17 Feb 2017, Julien Grall wrote:
> Hi,
> 
> On 02/17/2017 11:02 AM, Dario Faggioli wrote:
> > Just very quickly...
> > 
> > On Thu, 2017-02-16 at 15:07 -0800, Stefano Stabellini wrote:
> > > (XEN) Active queues: 1
> > > (XEN)   default-weight     = 256
> > > (XEN) Runqueue 0:
> > > (XEN)   ncpus              = 4
> > > (XEN)   cpus               = 0-3
> > > (XEN)   max_weight         = 256
> > > (XEN)   instload           = 1
> > > (XEN)   aveload            = 3208 (~1%)
> > > (XEN) l(XEN)    idlers: 00000000,00000000,00000000,0000000a
> > > a(XEN)  tickled: 00000000,00000000,00000000,00000000
> > > t(XEN)  fully idle cores: 00000000,00000000,00000000,0000000a
> > > e(XEN) Domain info:
> > > n(XEN)  Domain: 0 w 256 v 4
> > > c(XEN)    1: [0.0] flags=2 cpu=0 credit=10500000 [w=256] load=3170 (~1%)
> > > (XEN)     2: y[0.1] flags=0 cpu=1  credit=10500000 [w=256]( load=131072
> > > (~50%)
> > > (XEN)     3: n[0.2] flags=0 cpu=2s credit=10500000 [w=256]) load=131072
> > > (~50%):
> > > (XEN)     4:  [0.3] flags=0 cpu=3m credit=10500000 [w=256]a load=131072
> > > (~50%)x
> > > 
> > Status of vcpus 2, 3 and 4 is a bit weird. I'll think about it.
> > 
> > > =(XEN)  Domain: 1 w 256 v 1
> > > 1(XEN)    5: 1[1.0] flags=2 cpu=2 credit=9713074 [w=256] load=56 (~0%)
> > > (XEN) Runqueue info:
> > > 6(XEN) runqueue 0:
> > > 9(XEN) CPUs info:
> > > 0(XEN) CPU[00]   runq=0, sibling=00000000,00000000,00000000,00000001,
> > > wcore=00000000,00000000,00000000,00000001
> > > 
> > This tells me that nr_cpu_ids is very big (I think it tells it is 128,
> > i.e., ARM default), which means cpumask_t-s are huge.
> > 
> > What does `xl info' says. On my (x86) test box, it's like this:
> > 
> >  ...
> >  nr_cpus                : 16
> >  max_cpu_id             : 63
> >  ...
> > 
> > (and I have NR_CPUS=256, i.e., x86 the default).

Indeed I have 127


> > Cpumasks being bigger also means cpumask operation being slower, and
> > this matters quite a bit in Credit2, because we use cpumasks a lot (but
> > also in Credit1, because we use cpumasks a little less than in Credit2,
> > but still quite a bit).
> > 
> > Isn't there a way, on ARM, to figure out online that you're not going
> > to have 128 cpus in the platform?
> 
> It is just we never set nr_cpu_ids on ARM :/. There was a patch on the ML a
> while ago [1] but never got applied.
> 
> Stefano, I think the patch is still valid. Could you apply it?
> [1] https://patchwork.kernel.org/patch/8177261/

I pushed the patch


> It would probably be worth to do the benchmark again with this patch applied.

Unfortunately the numbers haven't changed much:

                                        AVG     MIN     MAX     WARM MAX
NODEBUG vwfi=sleep credit2 fix-cpumasks 8020    7670    10320   8390
NODEBUG vwfi=sleep credit1 fix-cpumasks 6400    6330    9650    6720

In addition to the mysterious difference between credit1 and credit2, we
also have the difference between vwfi=idle and vwfi=sleep to deal with:

NODEBUG vwfi=idle credit2 fix cpumasks  4000    2370    4500    3350
NODEBUG vwfi=idle credit1 fix cpumasks  3220    2180    4500    4320

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel