Re: [Xen-devel] xen does not support the 8G large bar

[Konrad Rzeszutek Wilk <konrad.wilk@xxxxxxxxxx>]

On August 10, 2016 11:30:37 PM EDT, "Gaofeng (GaoFeng, Euler)" 
<visen.gao@xxxxxxxxxx> wrote:
>Hi George,
>
>I found that you have submitted a patch about "libxl,hvmloader: Don't
>relocate memory for MMIO hole". So, I have a "8G large bar PCI
>device(NVIDIA Tesla M60)" passthrough question to ask you.
>
>
>
>Host passthrough PCI Device info:
>lspci -vs 06:00.0
>06:00.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
>        Subsystem: NVIDIA Corporation Device 115e
>        Flags: fast devsel, IRQ 40
>    Memory at 93000000 (32-bit, non-prefetchable) [disabled] [size=16M]
>      Memory at 23c00000000 (64-bit, prefetchable) [disabled] [size=8G]
>     Memory at 23e00000000 (64-bit, prefetchable) [disabled] [size=32M]
>        Capabilities: [60] Power Management version 3
>        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
>        Capabilities: [78] Express Endpoint, MSI 00
>        Capabilities: [100] Virtual Channel
>        Capabilities: [250] Latency Tolerance Reporting
>        Capabilities: [258] #1e
>        Capabilities: [128] Power Budgeting <?>
>        Capabilities: [420] Advanced Error Reporting
>Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024
><?>
>        Capabilities: [900] #19
>        Kernel driver in use: pciback
>        Kernel modules: nvidiafb
>
>Guest passthrough PCI Device info:
>lspci -vs 00:05.0
>00:05.0 3D controller: NVIDIA Corporation Device 13f2 (rev a1)
>        Subsystem: NVIDIA Corporation Device 115e
>        Physical Slot: 5
>        Flags: fast devsel, IRQ 36
>        Memory at f5000000 (32-bit, non-prefetchable) [size=16M]
>        Memory at 200000000 (64-bit, prefetchable) [size=4G]
>        Memory at f2000000 (64-bit, prefetchable) [size=32M]
>        Capabilities: [60] Power Management version 3
>        Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
>        Capabilities: [78] Express Endpoint, MSI 00
>        Kernel modules: nouveau, nvidiafb
>
>
>passthrough to the guest, the large bar size only has 4G. so, the
>passthrough pci device does not work, windows 7/2008 R2 64bit vm has
>the same problem.
>
>.
>
>According to the analysis hvmloader code, find a problem:
>
>
>
>if (is_64bar) {
>
>                bar_data_upper = pci_readl(devfn, bar_reg + 4);
>
>                pci_writel(devfn, bar_reg + 4, ~0);
>
>                bar_sz_upper = pci_readl(devfn, bar_reg + 4);
>
>                pci_writel(devfn, bar_reg + 4, bar_data_upper);
>
>                bar_sz = (bar_sz_upper << 32) | bar_sz;
>
>            }
>
>            bar_sz &= ~(bar_sz - 1);
>
>
>
>read from the pci device, bar_sz_upper is 0xffffffff, if the bar size
>is 8G, the bar_sz_upper should be 0xfffffffe.
>
>Have you ever encountered a similar problem?
>
>

Yes but only if you are using qemu-trad.

You need to have

https://github.com/xenserver/qemu-trad.pg/blob/master/master/0001-Add-64-bit-support-to-QEMU.patch

>
>Env info:
>
>device_model_version: qemu-xen
>
>xen version: xen-4.6.1
>
>Guest: RedHat-6.4-64
>
>Passthrough PCI Device: NVIDIA Tesla M60(compute mode)
>
>CPU: Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz
>
>
>
>Thanks
>
>Feng
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@xxxxxxxxxxxxx
>https://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel