Ed,
Can you try using the latest BIOS and the tip of xen-unstable?
Joe
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Nadolski, Ed
Sent: Wednesday, September 10, 2008 3:49 PM
To: Nadolski, Ed; Han, Weidong; Kumar, Venkat; Cui, Dexuan; Bryan York
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] VT-d issue on Xen 3.3.0 [WAS: Xen-3.2.1
VT-dSupport (NOT SURE WHETHER IT'S ABUGOR...)]
I have more info: when I
flash to an older BIOS, version 0559 (this required the recovery procedure) the
VT-d seems to come up OK. The xm dmesg is below.
Iâm surprised the old BIOS works
better. So be careful about upgrading your BIOS. Any thoughts on what
the bug might be? Is there anywhere I can report this to Intel?
Thanks,
Ed
(XEN) Xen EXPERIMENTAL TEST BUILD
(XEN) Xen version 3.3.0 (root@) (gcc
version 4.1.2 20071124 (Red Hat 4.1.2-42)) Wed Sep 10 14:43:01 MDT 2008
(XEN) Latest ChangeSet: unavailable
(XEN) Xen EXPERIMENTAL TEST BUILD!
(XEN) Command line: vtd=1 iommu=1
acpi=force apic=on
(XEN) Video information:
(XEN) VGA is text mode 80x25,
font 8x16
â
(XEN) Processor #0 6:15 APIC version
20
(XEN) Processor #1 6:15 APIC version 20
(XEN) IOAPIC[0]: apic_id 2, version
32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:
Flat. Using 1 I/O APICs
(XEN) [VT-D]dmar.c:446: Host address
width 36
(XEN) [VT-D]dmar.c:450: ED_DEBUG:
dmar:0xffff828bfff7b000
(XEN) [VT-D]dmar.c:451: ED_DEBUG:
entry_header:0xffff828bfff7b030
(XEN) [VT-D]dmar.c:452: ED_DEBUG:
table->length:296
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b030 type:0x0 length:24
(XEN) [VT-D]dmar.c:463: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb00000
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1b.0
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b048 type:0x0 length:32
(XEN) [VT-D]dmar.c:463: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb01000
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:2.1
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b068 type:0x0 length:48
(XEN) [VT-D]dmar.c:463: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323: dmaru->address
= feb02000
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:3.0
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:3.1
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:3.2
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:3.3
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b098 type:0x0 length:16
(XEN) [VT-D]dmar.c:463: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb03000
(XEN) [VT-D]dmar.c:332: found
INCLUDE_ALL
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b0a8 type:0x1 length:88
(XEN) [VT-D]dmar.c:467: found
ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:359: ED_DEBUG:
RMRR base:0x00000000000e0000 RMRR end:0x00000000000effff
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1d.0
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1d.1
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1d.7
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1a.0
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1a.1
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1a.2
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:1a.7
(XEN) [VT-D]dmar.c:458: ED_DEBUG:
entry_header:0xffff828bfff7b100 type:0x1 length:40
(XEN) [VT-D]dmar.c:467: found
ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:359: ED_DEBUG:
RMRR base:0x00000000cf600000 RMRR end:0x00000000cfffffff
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:2.0
(XEN) [VT-D]dmar.c:272: found
endpoint: bdf = 0:2.1
(XEN) Intel VT-d has been enabled
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Nadolski, Ed
Sent: Wednesday, September 10, 2008 1:56 PM
To: Han, Weidong; Kumar, Venkat; Cui, Dexuan; Bryan York
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] VT-d issue on Xen 3.3.0 [WAS: Xen-3.2.1 VT-d
Support (NOT SURE WHETHER IT'S ABUGOR...)]
I am seeing the âRMRR is
incorrectâ issue with Xen 3.3.0 and CentOS 5.2 on a DQ35 with the latest BIOS
(JOQ3510J.86A.0942.2008.0807.1958). It happens with either IGD or
PEG. CPU is E6750, main memory is 4Gb. Debug trace is below.
Iâm not sure what the RMRR entries
should look like. What BIOS revision worked for you? Should I
change any other BIOS settings? Current video settings are:
DVMT Mode = DVMT
IGD DVMT Memory = 128Mb
IGD Aperture Size = 256Mb
Are there any other things that
would be good to try?
Thanks,
Ed Nadolski
ed.nadolski@xxxxxxx
(XEN) Xen version 3.3.0 (root@) (gcc
version 4.1.2 20071124 (Red Hat 4.1.2-42)) Tue Sep 9 17:36:19 MDT 2008
(XEN) Latest ChangeSet: unavailable
(XEN) Xen EXPERIMENTAL TEST BUILD!
(XEN) Command line: vtd=1 iommu=1
acpi=force apic=on
(XEN) Video information:
(XEN) VGA is text mode 80x25,
font 8x16
(XEN) VBE/DDC methods: V2; EDID
transfer time: 1 seconds
ââ
(XEN) Processor #0 6:15 APIC version
20
(XEN) Processor #1 6:15 APIC version
20
(XEN) IOAPIC[0]: apic_id 2, version
32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:
Flat. Using 1 I/O APICs
(XEN) [VT-D]dmar.c:447: Host address
width 36
(XEN) [VT-D]dmar.c:456: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb00000
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1b.0
(XEN) [VT-D]dmar.c:456: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb01000
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:2.0
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:2.1
(XEN) [VT-D]dmar.c:456: found
ACPI_DMAR_DRHD
(XEN) [VT-D]dmar.c:323:
dmaru->address = feb03000
(XEN) [VT-D]dmar.c:332: found
INCLUDE_ALL
(XEN) [VT-D]dmar.c:460: found
ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR
base:0x00000000000e0000 RMRR end:0x00000000000effff
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1d.0
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1d.1
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1d.2
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1d.7
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1a.0
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1a.1
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1a.2
(XEN) [VT-D]dmar.c:272: found endpoint:
bdf = 0:1a.7
(XEN) [VT-D]dmar.c:460: found
ACPI_DMAR_RMRR
(XEN) [VT-D]dmar.c:359: ED_DEBUG: RMRR
base:0x0000000000000000 RMRR end:0x0000000000000000
(XEN) [VT-D]dmar.c:363: RMRR is
incorrect.
(XEN) Failed to parse ACPI DMAR. Disabling
VT-d.
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Han, Weidong
Sent: Wednesday, August 06, 2008 2:59 AM
To: Kumar, Venkat; Cui, Dexuan; Bryan York
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S
ABUGOR...)
Venkat,
I think you are using PEG (which
means IGD is disabled), right? This is a BIOS. I suggest you use IGD at
this moment.
Randy (Weidong)
From: Kumar, Venkat
[mailto:Venkat.Kumar@xxxxxxx]
Sent: 2008å8æ6æ
16:22
To: Cui, Dexuan; Bryan York; Han, Weidong
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S
ABUGOR...)
As Dexuan suggested I used
xen3.2.2-rc2.
With this version of xen with
vtd=1 boot parameter I could boot successfully (I was not able to boot with
vtd=1 in xen-3.2.1).
But I see the following in the
boot messages when I do a xm dmesg after a boot.
===================================================
(XEN) Xen heap: 14MB (14956kB)
(XEN) Domain heap initialised: DMA
width 32 bits
(XEN) Processor #0 6:15 APIC
version 20
(XEN) Processor #1 6:15 APIC
version 20
(XEN) IOAPIC[0]: apic_id 2,
version 32, address 0xfec00000, GSI 0-23
(XEN) Enabling APIC mode:
Flat. Using 1 I/O APICs
(XEN) [VT-D]dmar.c:441: RMRR is
incorrect.
(XEN) Failed to parse ACPI
DMAR. Disabling VT-d.
(XEN) [VT-D]ACPI DMAR:No DMAR
devices found
===================================================
If you observe the messages
highlighted in RED.
VT-d is getting disabled due to
some RMRR incorrect problem and in the dmar.c code this is the condition which
is causing this.
if (
rmrr->base_address >= rmrr->end_address )
{
dprintk(XENLOG_ERR VTDPREFIX, "RMRR is incorrect.\n");
return -EFAULT;
}
This code is in acpi_parse_one_rmrr function.
Thx,
Venkat
================================
Venkata Kumar Duvvuru,
LSI Engenio,
Adv. Development,
Bangalore.
Mob: +91-9880318542
Off : +91-80-41978700 ( Extn :
3544 )
================================
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
Sent: Wednesday, August 06, 2008 10:58 AM
To: Bryan York; Han, Weidong
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
BUGOR...)
Hi Bryan,
According to our tests using the latest
xen-unstable, the assignment of SATA controller (like in AHCI mode or
Enhanced mode) should be OK on the DQ35 platform.
For xen-unstable, I guess you didn't specify
"iommu=1" in the Xen grub entry? By default, VT-d is not enabled in
the latest xen-unstable.
From: Bryan York
[mailto:bryan.york@xxxxxxxxx]
Sent: 2008å8æ6æ
12:10
To: Han, Weidong
Cc: Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
BUGOR...)
Hello,
Thanks for the information. I'm trying to forward a PCI express SATA/PATA
controller with a JMicron chipset. (Info
here.) I know there have been problems with graphics cards in general, but
I thought it was possible with disk controllers? Is there a list of well
supported hardware (disk controllers) that can be used for VT-d?
Also, your RMRR explanation makes sense for the mercurial 3.2-testing, but what
about xen-unstable? I didn't see any errors there besides "I/O
Virtualisation disabled" with no other reason listed.
Thanks so much for your help on this. I will also try removing the disk
controller and see if the mercurial Xen-3.2-testing boots without RMRR errors.
Regards,
-Bryan
On Tue, Aug 5, 2008 at 6:27 PM, Han, Weidong
<weidong.han@xxxxxxxxx> wrote:
Hi Bryan,
I think you are using an add-on
graphic card (iGfx disabled), right? We also met this issue before. Obviously
it's a BIOS issue (RMRR is incorrect). When parsing apci dmar
failed, VT-d will be disabled.
Randy (Weidong)
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]
On Behalf Of Bryan York
Sent: 2008å8æ6æ
6:17
To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
BUGOR...)
Hello,
I'm
also seeing this exact same problem. I've posted on xen-users, but did not get
an answer. Venkat seems to be experiencing the same problem as me. I have the
latest BIOS for my motherboard DQ35JO. (BIOS ver 933) I'm running a Q6600 as
well. It hangs at "Brought up 4 CPUs." I've tried all sorts of
combinations of Xen versions and kernels to get this working. And, yes, VT and
VT-d are enabled in my BIOS.
1)
CentOS 5.2 SRPMS compiled with Xen 3.2.1.
Adding pci=nommconf does not get it to boot.
2)
Mercurial Xen-3.2-testing. Here, after adding pci=nommconf, I get:
(XEN)
[VT-D]dmar.c:441: RMRR is incorrect.
(XEN) Failed to parse ACPI DMAR. Disabling VT-d.
(XEN) [VT-D]ACPI DMAR:No DMAR devices found
3)
Mercurial Xen-unstable, I get "I/O Virtualisation disabled" I can't
find any reason why this is the case. VT-D is not enabled.
I
would really like to find the cause for this. I don't know if this is a BIOS
bug or what. This board is supposedly supported on Xen per the VT-D wiki. Other
people seem to have the same problem as me:
http://forums12.itrc.hp.com/service/forums/bizsupport/questionanswer.do?admit=109447627+1217898320366+28353475&threadId=1234588
Thanks
for any and all help,
-Bryan
The
BIOS version I use may be not the exact one as yours. I mean I expect you're
not using a very old BIOS. :-)
How
about trying the xen 3.2.2-rc2 and the latest xen-unstable first? There are
some VT-d bug fixes between 3.2.1 (you're using it) and 3.2.2-rc2.
Thanks,
--
Dexuan
________________________________
From:
Kumar, Venkat [mailto:Venkat.Kumar@xxxxxxx]
Sent: 2008å8æ5æ
14:37 To: Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx Subject: RE: [Xen-devel]
Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A BUG
OR...)
If
you don't mind can you send me the BIOS version you are using?
Thx,
Venkat
================================
Venkata
Kumar Duvvuru,
LSI
Engenio,
Adv.
Development,
Bangalore.
Mob:
+91-9880318542
Off
: +91-80-41978700 ( Extn : 3544 )
================================
________________________________
From:
Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx]
Sent: Tuesday, August 05, 2008 12:03 PM To: Kumar, Venkat; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A
BUG
OR...)
Hi
Venkat,
Can
you confirm you're using the latest BIOS? Can you try the xen 3.2.2-rc2 and
the latest xen-unstable on the same host?
I
have the similar host. Xen 3.2.2-rc2 and the latest xen-unstable both work
well on it.
Thanks,
--
Dexuan
________________________________
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]
On Behalf Of Kumar, Venkat Sent: 2008å8æ5æ
12:21 To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S A BUG
OR...)
After
building Xen-3.2.1 I could successfully boot into it if I avoid giving
vtd=1 as the boot parameter.
If
I pass vtd=1 as the boot parameter to xen, the system hangs while booting.
The
system hangs after the following messages
============================
Boot
messages
[XEN]â..
[Xen]
Brought up 2 CPUs...
=========================
My
chipset DQ35 series is having VT-d support and is enabled in the BIOS as
well.
My
domain0 is CentOS-5.2(2.6.18.8-xen).
Is
this a possible bug or some thing else??
Any
Idea??
Thx,
