Re: [Xen-devel] Xen 4.3 development update

[George Dunlap <George.Dunlap@xxxxxxxxxxxxx>]

On Thu, Apr 25, 2013 at 4:20 PM, George Dunlap
<George.Dunlap@xxxxxxxxxxxxx> wrote:
> On Thu, Apr 4, 2013 at 4:23 PM, Tim Deegan <tim@xxxxxxx> wrote:
>> At 11:34 -0400 on 03 Apr (1364988853), Andres Lagar-Cavilla wrote:
>>> On Apr 3, 2013, at 6:53 AM, George Dunlap <george.dunlap@xxxxxxxxxxxxx> 
>>> wrote:
>>>
>>> > On 03/04/13 08:27, Jan Beulich wrote:
>>> >>>>> On 02.04.13 at 18:34, Tim Deegan <tim@xxxxxxx> wrote:
>>> >>> This is a separate problem.  IIRC the AMD XP perf issue is caused by the
>>> >>> emulation of LAPIC TPR accesses slowing down with Andres's p2m locking
>>> >>> patches.  XP doesn't have 'lazy IRQL' or support for CR8, so it takes a
>>> >>> _lot_ of vmexits for IRQL reads and writes.
>>> >> Ah, okay, sorry for mixing this up. But how is this a regression
>>> >> then?
>>> >
>>> > My sense, when I looked at this back whenever that there was much more to 
>>> > this.  The XP IRQL updating is a problem, but it's made terribly worse by 
>>> > the changset in question.  It seemed to me like the kind of thing that 
>>> > would be caused by TLB or caches suddenly becoming much less effective.
>>>
>>> The commit in question does not add p2m mutations, so it doesn't nuke the 
>>> NPT/EPT TLBs. It introduces a spin lock in the hot path and that is the 
>>> problem. Later in the 4.2 cycle we changed the common case to use an 
>>> rwlock. Does the same perf degradation occur with tip of 4.2?
>>>
>>
>> Yes, 4.2 is definitely slower.  A compile test on a 4-vcpu VM that takes
>> about 12 minutes before this locking change takes more than 20 minutes
>> on the current tip of xen-unstable (I gave up at 22 minutes and rebooted
>> to test something else).
>
> Tim,
>
> Can you go into a bit more detail about what you complied on what kind of OS?
>
> I just managed to actually find a c/s from which I could build the
> tools (git 914e61c), and then compared that with just rebuilding xen
> on accused changeset (6b719c3).
>
> The VM was a Debian Wheezy VM, stock kernel (3.2), PVHVM mode, 1G of
> RAM, 4 vcpus, LVM-backed 8G disk.
>
> Host is an AMD Barcelona (I think), 8 cores, 4G RAM.
>
> The test was "make -C xen clean && make -j 6 XEN_TARGET_ARCH=x86_64 xen".
>
> Time was measured on the "test controller" machine -- i.e., my dev
> box, which is not running Xen.  (This means there's some potential for
> timing variance with ssh and the network, but no potential for timing
> variance due to virtual time issues.)
>
> "Good" (c/s 914e61c):
> 334.92
> 312.22
> 311.21
> 311.71
> 315.87
>
> "Bad" (c/s 6b719c3)
> 326.50
> 295.77
> 288.50
> 296.43
> 276.66
>
> In the "Good" run I had a vnc display going, whereas in the "bad" run
> I didn't; that could account for the speed-up.  But so far it
> contradicts the idea of a systematic problem in c/s 6b719c3.
>
> I'm going to try some other combinations as well...

BTW, I did the same test with 4.1, 4.2.2-RC2, and a recent
xen-unstable tip.  Here are all the results, presented in the order of
the version of xen tested:

v4.1:
292.35
267.31
270.91
285.81
278.30

"Good" git c/s 914e61c:
334.92
312.22
311.21
311.71
315.87

"Bad" git c/s 6b719c3:
326.50
295.77
288.50
296.43
276.66

4.2.2-rc2:
261.49
250.75
246.82
246.23
247.64

Xen-unstable "recent" master:
267.31
258.49
256.83
250.77
252.36

So overall I think we can say that c/s 6b719c3 didn't cause a general
performance regression on AMD HVM guests.

I'm in the process of duplicating the exact test which Peter noticed,
namely time to boot Windows XP.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel