[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] [xen-unstable test] 65141: regressions - FAIL



On Wed, 2015-12-02 at 13:51 +0000, Ian Campbell wrote:

> http://osstest.test-lab.xenproject.org/~osstest/pub/logs/65301/
> 
> I think that ought to give a baseline for the bisector to work with. I'll
> prod it to do so.

Results are below. TL;DR: d02e84b9d9d "vVMX: use latched VMCS machine
address" is somehow at fault.

It appears to be somewhat machine specific, the one this has been
failing on is godello* which says "CPU0: Intel(R) Xeon(R) CPU E3-1220
v3 @ 3.10GHz stepping 03" in its serial log.

Andy suggested this might be related to cpu_has_vmx_vmcs_shadowing
so Haswell and newer vs IvyBridge and older.

Ian.

branch xen-unstable
xenbranch xen-unstable
job test-amd64-amd64-qemuu-nested-intel
testid debian-hvm-install/l1/l2

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
  Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/


  commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
  Author: Jan Beulich <jbeulich@xxxxxxxx>
  Date:   Tue Nov 24 12:07:27 2015 +0100
  
      vVMX: use latched VMCS machine address
      
      Instead of calling domain_page_map_to_mfn() over and over, latch the
      guest VMCS machine address unconditionally (i.e. independent of whether
      VMCS shadowing is supported by the hardware).
      
      Since this requires altering the parameters of __[gs]et_vmcs{,_real}()
      (and hence all their callers) anyway, take the opportunity to also drop
      the bogus double underscores from their names (and from
      __[gs]et_vmcs_virtual() as well).
      
      Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
      Acked-by: Kevin Tian <kevin.tian@xxxxxxxxx>


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.html
Revision IDs in each graph node refer, respectively, to the Trees above.

----------------------------------------
Running cs-bisection-step 
--graph-out=/home/logs/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2
 --summary-out=tmp/65388.bisection-summary 
--blessings=real,real-bisect,adhoc-bisect --basis-template=65301 
--basis-flight=65301 xen-unstable test-amd64-amd64-qemuu-nested-intel 
debian-hvm-install/l1/l2
Searching for failure / basis pass:
 65314 fail [host=godello1] / template as basis? using template as basis.
Failure / basis pass flights: 65314 / 65301
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
2c4f313a7e62c7e559a469d4af4c3d03c49afa43
Basis pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
d07f63fa6e70350b23e7acbde06129247c4e655d
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9-769b79eb206ad5b0249a08665fefb913c3d1998e
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#bc00cad75d8bcc3ba696992bec219c21db8406aa-bc00cad75d8bcc3ba696992bec219c21db8406aa
 
git://xenbits.xen.org/qemu-xen.git#816609b2841297925a223ec377c336360e044ee5-3fb401edbd8e9741c611bfddf6a2032ca91f55ed
 
git://xenbits.xen.org/xen.git#d07f63fa6e70350b23e7acbde06129247c4e655d-2c4f313a7e62c7e559a469d4af4c3d03c49afa43
Loaded 17133 nodes in revision graph
Searching for test results:
 65114 [host=italia0]
 65141 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
b1d398b67781140d1c6efd05778d0ad4103b2a32
 65162 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
713b7e4ef2aa4ec3ae697cde9c81d5a57548f9b1
 65164 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
b1d398b67781140d1c6efd05778d0ad4103b2a32
 65186 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
b1d398b67781140d1c6efd05778d0ad4103b2a32
 65217 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
b1d398b67781140d1c6efd05778d0ad4103b2a32
 65233 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
b1d398b67781140d1c6efd05778d0ad4103b2a32
 65287 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
c8eb0ec277ae387e78d685523e0fee633e46f046
 65333 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
d07f63fa6e70350b23e7acbde06129247c4e655d
 65314 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
2c4f313a7e62c7e559a469d4af4c3d03c49afa43
 65354 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
c35eefded2992fc9b979f99190422527650872fd
 65267 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
4c6cd64519f9bc270a7278128c94e4b66e3d2077
 65301 pass 1230ae0e99e05ced8a945a1a2c5762ce5c6c97c9 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
d07f63fa6e70350b23e7acbde06129247c4e655d
 65351 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
0ba0faa7f02fb310a330579adeef3534e6eca5a8
 65343 pass 997badf1e03dfd39854094f7767e1a4cf5ed310b 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
d07f63fa6e70350b23e7acbde06129247c4e655d
 65325 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
c8eb0ec277ae387e78d685523e0fee633e46f046
 65345 pass e7a10d9297c1abfd27138d86a13f7b6435634a46 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
d07f63fa6e70350b23e7acbde06129247c4e655d
 65341 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
2c4f313a7e62c7e559a469d4af4c3d03c49afa43
 65342 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
c87303c04738b0e837da6e891eb561de0bf1b64e
 65362 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
 65367 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
816609b2841297925a223ec377c336360e044ee5 
827db7b26384ce083df7154d77f13379b2cf4121
 65370 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
827db7b26384ce083df7154d77f13379b2cf4121
 65374 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
3b47431691409004c7218f6a6ba5c9c0bcf483ea
 65379 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
 65380 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
3b47431691409004c7218f6a6ba5c9c0bcf483ea
 65383 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
 65384 pass 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
3b47431691409004c7218f6a6ba5c9c0bcf483ea
 65388 fail 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
Searching for interesting versions
 Result found: flight 65301 (pass), for basis pass
 Result found: flight 65314 (fail), for basis failure
 Repro found: flight 65333 (pass), for basis pass
 Repro found: flight 65341 (fail), for basis failure
 0 revisions at 769b79eb206ad5b0249a08665fefb913c3d1998e 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
bc00cad75d8bcc3ba696992bec219c21db8406aa 
3fb401edbd8e9741c611bfddf6a2032ca91f55ed 
3b47431691409004c7218f6a6ba5c9c0bcf483ea
No revisions left to test, checking graph state.
 Result found: flight 65374 (pass), for last pass
 Result found: flight 65379 (fail), for first failure
 Repro found: flight 65380 (pass), for last pass
 Repro found: flight 65383 (fail), for first failure
 Repro found: flight 65384 (pass), for last pass
 Repro found: flight 65388 (fail), for first failure

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
  Bug not present: 3b47431691409004c7218f6a6ba5c9c0bcf483ea
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/65388/


  commit d02e84b9d9d16b6b56186f0dfdcb3c90b83c82a3
  Author: Jan Beulich <jbeulich@xxxxxxxx>
  Date:   Tue Nov 24 12:07:27 2015 +0100
  
      vVMX: use latched VMCS machine address
      
      Instead of calling domain_page_map_to_mfn() over and over, latch the
      guest VMCS machine address unconditionally (i.e. independent of whether
      VMCS shadowing is supported by the hardware).
      
      Since this requires altering the parameters of __[gs]et_vmcs{,_real}()
      (and hence all their callers) anyway, take the opportunity to also drop
      the bogus double underscores from their names (and from
      __[gs]et_vmcs_virtual() as well).
      
      Signed-off-by: Jan Beulich <jbeulich@xxxxxxxx>
      Acked-by: Kevin Tian <kevin.tian@xxxxxxxxx>

Revision graph left in 
/home/logs/results-adhoc/bisect/xen-unstable/test-amd64-amd64-qemuu-nested-intel.debian-hvm-install--l1--l2.{dot,ps,png,html,svg}.
----------------------------------------
65388: tolerable ALL FAIL

flight 65388 xen-unstable adhoc-bisect [real]
http://logs.test-lab.xenproject.org/osstest/logs/65388/

Failures :-/ but no regressions.

Tests which did not succeed,
including tests which could not be run:
 test-amd64-amd64-qemuu-nested-intel 16 debian-hvm-install/l1/l2 fail baseline 
untested


jobs:
 test-amd64-amd64-qemuu-nested-intel                          fail    


------------------------------------------------------------
sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
    http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
    http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
    http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.