Re: [Xen-devel] Delays on usleep calls

[Pavlo Suikov <pavlo.suikov@xxxxxxxxxxxxxxx>]

Hi again,

sorry for the broken formatting, see better one below where the tables should be.

> x86 or ARM host?

ARM. ARMv7, TI Jacinto6 to be precise.

> Also, how many pCPUs and vCPUs do the host and the various guests have?

2 pCPUs, 4 vCPUs: 2 vCPU per domain.

> # xl list -n
> # xl vcpu-list

# xl list -n
Name                                        ID   Mem VCPUs      State   Time(s) NODE Affinity
Domain-0                                     0   118     2     r-----      20.5 any node
android_4.3                                  1  1024     2     -b----     383.5 any node

# xl vcpu-list
Name                                ID  VCPU   CPU State   Time(s) CPU Affinity
Domain-0                             0     0    0   r--      12.6  any cpu
Domain-0                             0     1    1   b       7.3  any cpu
android_4.3                          1     0    0   b     180.8  any cpu
android_4.3                          1     1    0   b     184.4  any cpu

> Are you using any vCPU-to-pCPU pinning?

No.

> What about giving a try to it yourself? I think standardizing on one (a
> set of) specific tool could be a good thing.

Yep, we'll try it.

> Mmm... Can you show us at least part of the numbers? From my experience,
> it's really easy to mess up with terminology in this domain (delay,
> latency, jitter, etc.).

Test with 30 ms sleep period gives such results (I hope formatting won't break):

                             Measurements  Average Number of times with t > 32 ms

Credit        Dom0             358             32.5                         72

                 DomU            358             40.7                        358

sEDF        Dom0             358             33.6                        120

                 DomU            358             40.7                        358

We did additional measurements and as you can see, my first impression was not very correct: difference between dom0 and domU exist and is quite observable on a larger scale. On the same setup bare metal without Xen number of times t > 32 is close to 0; on the setup with Xen but without domU system running number of times t > 32 is close to 0 as well.  We will make additional measurements with Linux (not Android) as a domU guest, though.

Raw data looks like this:

Credit:

Dom0    DomU

38            46
31            46
31            46
39            46
39            46
31            46
31            39

sEDF:

Dom0    DomU
38            39
39            39
39            39
39            46
39            46
31            39
31            39

So as you can see, there are peak values both in dom0 and domU.

> AFAIUI, you're saying that you're asking for a sleep time of X, and
> you're being waking up in the interval [x+5ms, x+15ms], is that the
> case?

Yes, that's correct. In the numbers above sleep period is 30 ms, so we were expecting 30-31 ms sleep time as it is on the same setup without Xen.

> # xl sched-sedf

# xl sched-sedf
Cpupool Pool-0:
Name                                ID Period Slice  Latency Extra Weight
Domain-0                             0    100      0       0     1      0
android_4.3                          1    100      0       0     1      0

> Or you have some other load in the system while
> performing these measurements? If the latter, what and where?

During this test both guests were almost idle.

> Oh, and now that I think about it, something that present in credit and
> not in sEDF that might be worth checking is the scheduling rate limiting
> thing.

We'll check it out, thanks!

Regards, Pavlo

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel