I have more info:Â when I flash to an
older BIOS, version 0559 (this required the recovery procedure) the VT-d seems
to come up OK. The xm dmesg is below.
Iâm surprised the old BIOS works better. So
be careful about upgrading your BIOS. Any thoughts on what the bug might be?Â
Is there anywhere I can report this to Intel?
Thanks,
Ed
(XEN) Xen
EXPERIMENTAL TEST BUILD
(XEN) Xen
version 3.3.0 (root@) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) Wed Sep
10 14:43:01 MDT 2008
(XEN) Latest
ChangeSet: unavailable
(XEN) Xen
EXPERIMENTAL TEST BUILD!
(XEN) Command
line: vtd=1 iommu=1 acpi=force apic=on
(XEN) Video
information:
(XEN)Â VGA is
text mode 80x25, font 8x16
â
(XEN)
Processor #0 6:15 APIC version 20
(XEN)
Processor #1 6:15 APIC version 20
(XEN)
IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN)
Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN)
[VT-D]dmar.c:446: Host address width 36
(XEN)
[VT-D]dmar.c:450: ED_DEBUG: dmar:0xffff828bfff7b000
(XEN)
[VT-D]dmar.c:451: ED_DEBUG: entry_header:0xffff828bfff7b030
(XEN)
[VT-D]dmar.c:452: ED_DEBUG: table->length:296
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b030 type:0x0 length:24
(XEN)
[VT-D]dmar.c:463: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb00000
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1b.0
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b048 type:0x0 length:32
(XEN)
[VT-D]dmar.c:463: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb01000
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b068 type:0x0 length:48
(XEN)
[VT-D]dmar.c:463: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb02000
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:3.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:3.1
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:3.2
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:3.3
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b098 type:0x0 length:16
(XEN)
[VT-D]dmar.c:463: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb03000
(XEN)
[VT-D]dmar.c:332: found INCLUDE_ALL
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b0a8 type:0x1 length:88
(XEN)
[VT-D]dmar.c:467: found ACPI_DMAR_RMRR
(XEN)
[VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x00000000000e0000Â RMRR
end:0x00000000000effff
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.1
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.2
(XEN) [VT-D]dmar.c:272:
found endpoint: bdf = 0:1d.7
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.1
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.2
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.7
(XEN)
[VT-D]dmar.c:458: ED_DEBUG: entry_header:0xffff828bfff7b100 type:0x1 length:40
(XEN)
[VT-D]dmar.c:467: found ACPI_DMAR_RMRR
(XEN)
[VT-D]dmar.c:359: ED_DEBUG: RMRR base:0x00000000cf600000Â RMRR
end:0x00000000cfffffff
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
(XEN) Intel VT-d has been enabled
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Nadolski, Ed
Sent: Wednesday, September 10,
2008 1:56 PM
To: Han, Weidong; Kumar, Venkat; Cui, Dexuan; Bryan York
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] VT-d issue on
Xen 3.3.0 [WAS: Xen-3.2.1 VT-d Support (NOT SURE WHETHER IT'S ABUGOR...)]
I am seeing the âRMRR is incorrectâ issue
with Xen 3.3.0 and CentOS 5.2 on a DQ35 with the latest BIOS
(JOQ3510J.86A.0942.2008.0807.1958). It happens with either IGD or
PEG. CPU is E6750, main memory is 4Gb. Debug trace is below.
Iâm not sure what the RMRR entries should
look like. What BIOS revision worked for you? Should I change any
other BIOS settings? Current video settings are:
DVMT Mode = DVMT
IGD DVMT Memory = 128Mb
IGD Aperture Size = 256Mb
Are there any other things that would be
good to try?
Thanks,
Ed Nadolski
ed.nadolski@xxxxxxx
(XEN) Xen
version 3.3.0 (root@) (gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)) Tue
Sep 9 17:36:19 MDT 2008
(XEN) Latest
ChangeSet: unavailable
(XEN) Xen
EXPERIMENTAL TEST BUILD!
(XEN) Command
line: vtd=1 iommu=1 acpi=force apic=on
(XEN) Video
information:
(XEN)
VGA is text mode 80x25, font 8x16
(XEN)
VBE/DDC methods: V2; EDID transfer time: 1 seconds
ââ
(XEN) Processor
#0 6:15 APIC version 20
(XEN)
Processor #1 6:15 APIC version 20
(XEN)
IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-23
(XEN)
Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN)
[VT-D]dmar.c:447: Host address width 36
(XEN) [VT-D]dmar.c:456:
found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb00000
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1b.0
(XEN)
[VT-D]dmar.c:456: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb01000
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:2.1
(XEN)
[VT-D]dmar.c:456: found ACPI_DMAR_DRHD
(XEN)
[VT-D]dmar.c:323: dmaru->address = feb03000
(XEN)
[VT-D]dmar.c:332: found INCLUDE_ALL
(XEN)
[VT-D]dmar.c:460: found ACPI_DMAR_RMRR
(XEN)
[VT-D]dmar.c:359: ED_DEBUG: RMRR
base:0x00000000000e0000 RMRR end:0x00000000000effff
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.1
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.2
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1d.7
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.0
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.1
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.2
(XEN)
[VT-D]dmar.c:272: found endpoint: bdf = 0:1a.7
(XEN)
[VT-D]dmar.c:460: found ACPI_DMAR_RMRR
(XEN)
[VT-D]dmar.c:359: ED_DEBUG: RMRR
base:0x0000000000000000 RMRR end:0x0000000000000000
(XEN)
[VT-D]dmar.c:363: RMRR is incorrect.
(XEN) Failed to
parse ACPI DMAR. Disabling VT-d.
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Han, Weidong
Sent: Wednesday, August 06, 2008
2:59 AM
To: Kumar,
Venkat; Cui, Dexuan; Bryan York
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1
VT-d Support (NOT SURE WHETHER IT'S ABUGOR...)
Venkat,
I think you are using PEG (which means IGD
is disabled), right? This is a BIOS. I suggest you use IGD at this moment.
Randy (Weidong)
From: Kumar, Venkat [mailto:Venkat.Kumar@xxxxxxx]
Sent: 2008å8æ6æ 16:22
To: Cui, Dexuan; Bryan York; Han,
Weidong
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1
VT-d Support (NOT SURE WHETHER IT'S ABUGOR...)
As Dexuan suggested I used xen3.2.2-rc2.
With this version of xen with vtd=1 boot
parameter I could boot successfully (I was not able to boot with vtd=1 in
xen-3.2.1).
But I see the following in the boot
messages when I do a xm dmesg after a boot.
===================================================
(XEN) Xen heap: 14MB (14956kB)
(XEN) Domain heap initialised: DMA width
32 bits
(XEN) Processor #0 6:15 APIC version 20
(XEN) Processor #1 6:15 APIC version 20
(XEN) IOAPIC[0]: apic_id 2, version 32,
address 0xfec00000, GSI 0-23
(XEN) Enabling APIC
mode: Flat. Using 1 I/O APICs
(XEN) [VT-D]dmar.c:441:
RMRR is incorrect.
(XEN) Failed to parse ACPI
DMAR. Disabling VT-d.
(XEN) [VT-D]ACPI DMAR:No
DMAR devices found
===================================================
If you observe the messages highlighted in
RED.
VT-d is getting disabled due to some RMRR
incorrect problem and in the dmar.c code this is the condition which is causing
this.
if (
rmrr->base_address >= rmrr->end_address )
{
dprintk(XENLOG_ERR VTDPREFIX, "RMRR is incorrect.\n");
return -EFAULT;
}
This code is in acpi_parse_one_rmrr function.
Thx,
Venkat
================================
Venkata
Kumar Duvvuru,
LSI
Engenio,
Adv.
Development,
Bangalore.
Mob:
+91-9880318542
Off :
+91-80-41978700 ( Extn : 3544 )
================================
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Cui, Dexuan
Sent: Wednesday, August 06, 2008
10:58 AM
To: Bryan York; Han, Weidong
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1
VT-d Support (NOT SURE WHETHER IT'S A BUGOR...)
Hi Bryan,
According to our tests using the latest xen-unstable, the
assignment of SATA controller (like in AHCI mode or Enhanced
mode) should be OK on the DQ35 platform.
For xen-unstable, I guess you didn't specify
"iommu=1" in the Xen grub entry? By default, VT-d is not enabled in
the latest xen-unstable.
From: Bryan York
[mailto:bryan.york@xxxxxxxxx]
Sent: 2008å8æ6æ 12:10
To: Han, Weidong
Cc: Cui, Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: Re: [Xen-devel] Xen-3.2.1
VT-d Support (NOT SURE WHETHER IT'S A BUGOR...)
Hello,
Thanks for the information. I'm trying to forward a PCI express SATA/PATA
controller with a JMicron chipset. (Info
here.) I know there have been problems with graphics cards in general, but
I thought it was possible with disk controllers? Is there a list of well
supported hardware (disk controllers) that can be used for VT-d?
Also, your RMRR explanation makes sense for the mercurial 3.2-testing, but what
about xen-unstable? I didn't see any errors there besides "I/O
Virtualisation disabled" with no other reason listed.
Thanks so much for your help on this. I will also try removing the disk
controller and see if the mercurial Xen-3.2-testing boots without RMRR errors.
Regards,
-Bryan
On Tue, Aug 5, 2008 at 6:27 PM, Han, Weidong <weidong.han@xxxxxxxxx> wrote:
Hi Bryan,
I think you are using an add-on graphic
card (iGfx disabled), right? We also met this issue before. Obviously it's a
BIOS issue (RMRR is incorrect). When parsing apci dmar
failed, VT-d will be disabled.
Randy (Weidong)
From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]
On Behalf Of Bryan York
Sent: 2008å8æ6æ 6:17
To: Cui, Dexuan
Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1
VT-d Support (NOT SURE WHETHER IT'S A BUGOR...)
Hello,
I'm
also seeing this exact same problem. I've posted on xen-users, but did not get
an answer. Venkat seems to be experiencing the same problem as me. I have the
latest BIOS for my motherboard DQ35JO. (BIOS ver 933) I'm running a Q6600 as
well. It hangs at "Brought up 4 CPUs." I've tried all sorts of combinations
of Xen versions and kernels to get this working. And, yes, VT and VT-d are
enabled in my BIOS.
1)
CentOS 5.2 SRPMS compiled with Xen 3.2.1.
Adding pci=nommconf does not get it to boot.
2)
Mercurial Xen-3.2-testing. Here, after adding pci=nommconf, I get:
(XEN)
[VT-D]dmar.c:441: RMRR is incorrect.
(XEN) Failed to parse ACPI DMAR. Disabling VT-d.
(XEN) [VT-D]ACPI DMAR:No DMAR devices found
3)
Mercurial Xen-unstable, I get "I/O Virtualisation disabled" I can't
find any reason why this is the case. VT-D is not enabled.
I
would really like to find the cause for this. I don't know if this is a BIOS
bug or what. This board is supposedly supported on Xen per the VT-D wiki. Other
people seem to have the same problem as me:
http://forums12.itrc.hp.com/service/forums/bizsupport/questionanswer.do?admit=109447627+1217898320366+28353475&threadId=1234588
Thanks
for any and all help,
-Bryan
The
BIOS version I use may be not the exact one as yours. I mean I expect you're
not using a very old BIOS. :-)
How
about trying the xen 3.2.2-rc2 and the latest xen-unstable first? There are
some VT-d bug fixes between 3.2.1
(you're using it) and 3.2.2-rc2.
Thanks,
--
Dexuan
________________________________
From:
Kumar, Venkat [mailto:Venkat.Kumar@xxxxxxx]
Sent: 2008å8æ5æ 14:37 To: Cui,
Dexuan; xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: RE: [Xen-devel] Xen-3.2.1 VT-d
Support (NOT SURE WHETHER IT'S A BUG
OR...)
If
you don't mind can you send me the BIOS version you are using?
Thx,
Venkat
================================
Venkata
Kumar Duvvuru,
LSI
Engenio,
Adv.
Development,
Bangalore.
Mob:
+91-9880318542
Off
: +91-80-41978700 ( Extn : 3544 )
================================
________________________________
From:
Cui, Dexuan [mailto:dexuan.cui@xxxxxxxxx]
Sent: Tuesday, August 05, 2008 12:03 PM To: Kumar,
Venkat; xen-devel@xxxxxxxxxxxxxxxxxxx Subject: RE: [Xen-devel]
Xen-3.2.1 VT-d Support (NOT SURE
WHETHER IT'S A BUG
OR...)
Hi
Venkat,
Can
you confirm you're using the latest BIOS? Can you try the xen 3.2.2-rc2 and
the latest xen-unstable on the same host?
I
have the similar host. Xen 3.2.2-rc2 and the latest xen-unstable both work
well on it.
Thanks,
--
Dexuan
________________________________
From:
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
[mailto:xen-devel-bounces@xxxxxxxxxxxxxxxxxxx]
On Behalf Of Kumar, Venkat Sent:
2008å8æ5æ 12:21 To: xen-devel@xxxxxxxxxxxxxxxxxxx
Subject: [Xen-devel] Xen-3.2.1 VT-d
Support (NOT SURE WHETHER IT'S A BUG OR...)
After
building Xen-3.2.1 I could successfully boot into it if I avoid giving
vtd=1 as the boot parameter.
If
I pass vtd=1 as the boot parameter to xen, the system hangs while booting.
The
system hangs after the following messages
============================
Boot
messages
[XEN]â..
[Xen]
Brought up 2 CPUs...
=========================
My
chipset DQ35 series is having VT-d
support and is enabled in the BIOS as
well.
My
domain0 is CentOS-5.2(2.6.18.8-xen).
Is
this a possible bug or some thing else??
Any
Idea??
Thx,