|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] DomU sees only 512MB RAM with PCI-passthrough
On Thu, 2012-12-13 at 11:43 +0000, Christian Holpert wrote:
> Hello Ian,
>
> At 10:38 13.12.2012, you wrote:
> >On Wed, 2012-12-12 at 17:04 +0000, Christian Holpert wrote:
> > > Please tell me, if you need more logs.
> >
> >dmesg of the guest booting both with and without the pci device might be
> >interesting. So would the output of "xl -vvv create" in both cases.
>
> attached.
In the diff of the dmesg I see:
--- dmesg_without_pci.log 2012-12-13 11:50:00.000000000 +0000
+++ dmesg_with_pci.log 2012-12-13 11:50:00.000000000 +0000
@@ -1,72 +1,92 @@
Reserving virtual address space above 0xf5800000
Linux version 3.6.6-gentoo (root@xen) (gcc version 4.5.4 (Gentoo 4.5.4
p1.0, pie-0.4.7) ) #1 SMP Tue Nov 13 17:47:58 CET 2012
ACPI in unprivileged domain disabled
+Freeing 20000-80000 pfn range: 393216 pages freed
+1-1 mapping on 20000->100000
+Released 393216 pages of unused memory
+Set 917504 page(s) to 1-1 mapping
e820: BIOS-provided physical RAM map:
Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
-Xen: [mem 0x0000000000100000-0x000000007fffffff] usable
+Xen: [mem 0x0000000000100000-0x000000001fffffff] usable
+Xen: [mem 0x0000000020000000-0x00000000201fffff] reserved
+Xen: [mem 0x0000000020200000-0x0000000040003fff] unusable
+Xen: [mem 0x0000000040004000-0x0000000040004fff] reserved
+Xen: [mem 0x0000000040005000-0x00000000d9cbefff] unusable
+Xen: [mem 0x00000000d9cbf000-0x00000000da285fff] reserved
+Xen: [mem 0x00000000da286000-0x00000000da505fff] ACPI NVS
+Xen: [mem 0x00000000da506000-0x00000000da50afff] ACPI data
+Xen: [mem 0x00000000da50b000-0x00000000da54dfff] ACPI NVS
+Xen: [mem 0x00000000da54e000-0x00000000dad55fff] unusable
+Xen: [mem 0x00000000dad56000-0x00000000daff1fff] reserved
+Xen: [mem 0x00000000daff2000-0x00000000daffffff] unusable
+Xen: [mem 0x00000000db800000-0x00000000df9fffff] reserved
+Xen: [mem 0x00000000f8000000-0x00000000fbffffff] reserved
+Xen: [mem 0x00000000fec00000-0x00000000fec00fff] reserved
+Xen: [mem 0x00000000fed00000-0x00000000fed03fff] reserved
+Xen: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
+Xen: [mem 0x00000000fee00000-0x00000000fee00fff] reserved
+Xen: [mem 0x00000000ff000000-0x00000000ffffffff] reserved
NX (Execute Disable) protection: active
MPS support code is not built-in.
Using acpi=off or acpi=noirq or pci=noacpi may have problem
DMI not present or invalid.
e820: update [mem 0x00000000-0x0000ffff] usable ==> reserved
e820: remove [mem 0x000a0000-0x000fffff] usable
-e820: last_pfn = 0x80000 max_arch_pfn = 0x1000000
+e820: last_pfn = 0x20000 max_arch_pfn = 0x1000000
On the libxl side with pci I also see:
+libxl: debug: libxl_pci.c:85:libxl__create_pci_backend: Creating pci
backend
+libxl: debug: libxl_x86.c:82:e820_sanitize: Memory: 2097152kB End of
RAM: 0x20000 (PFN) Delta: 1572864kB, PCI start: 524288kB (0x20000 PFN), Balloon
0kB
+
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [0 -> 20000] RAM
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [20000 -> 20200]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [20200 -> 40004]
Unusable
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [40004 -> 40005]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [40005 -> d9cbf]
Unusable
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [d9cbf -> da286]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [da286 -> da506] ACPI
NVS
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [da506 -> da50b] ACPI
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [da50b -> da54e] ACPI
NVS
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [da54e -> dad56]
Unusable
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [dad56 -> daff2]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [daff2 -> db000]
Unusable
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [db800 -> dfa00]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [f8000 -> fc000]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [fec00 -> fec01]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed00 -> fed04]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [fed1c -> fed20]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [fee00 -> fee01]
Reserved
+libxl: debug: libxl_x86.c:201:e820_sanitize: : [ff000 -> 100000]
Reserved
So I suspect this is some interaction between xl's e820_host option
(which gets automatically enabled if you give a PCI device) and the
kernels early memory layout stuff. I'm CCing David and Konrad in the
hopes they have an idea what is happening.
Ian.
>
> >Is the 512MB number coming only from within the guest or is it also
> >visible in e.g. xl list?
>
> with pci:
> xen xen # xl list
> Name ID Mem VCPUs State
> Time(s)
> Domain-0 0 2557 4 r-----
> 2928.2
> nodi 27 512 3 r-----
> 34.1
>
> --------------
>
> without pci:
> xen xen # xl list
> Name ID Mem VCPUs State
> Time(s)
> Domain-0 0 2557 4 r-----
> 2953.2
> nodi 28 2048 3 -b----
> 0.6
>
> >You could try ballooning up inside the guest,
> >by writing to the /sys/devices/system/xen_memory/xen_memory0/target_kb
> >sysfs node.
>
> (with pci:)
> No effect. sysfs already shows 2GB, but is ignored for values above 512MB.
>
> nodi ~ # cat /sys/devices/system/xen_memory/xen_memory0/target_kb
> 2097152
> nodi ~ # cat /proc/meminfo
> MemTotal: 511268 kB
> MemFree: 266328 kB
> Buffers: 16572 kB
> Cached: 152604 kB
> SwapCached: 0 kB
> Active: 65400 kB
> Inactive: 157904 kB
> Active(anon): 54180 kB
> Inactive(anon): 332 kB
> Active(file): 11220 kB
> Inactive(file): 157572 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> HighTotal: 0 kB
> HighFree: 0 kB
> LowTotal: 511268 kB
> LowFree: 266328 kB
> SwapTotal: 983036 kB
> SwapFree: 983036 kB
> Dirty: 0 kB
> Writeback: 0 kB
> AnonPages: 54208 kB
> Mapped: 33852 kB
> Shmem: 388 kB
> Slab: 8552 kB
> SReclaimable: 3284 kB
> SUnreclaim: 5268 kB
> KernelStack: 760 kB
> PageTables: 1076 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 1238668 kB
> Committed_AS: 237812 kB
> VmallocTotal: 339960 kB
> VmallocUsed: 7816 kB
> VmallocChunk: 331464 kB
> DirectMap4k: 524288 kB
> DirectMap2M: 0 kB
> nodi ~ # echo 1048576 > /sys/devices/system/xen_memory/xen_memory0/target_kb
> nodi ~ # cat /sys/devices/system/xen_memory/xen_memory0/target_kb
> 1048576
> nodi ~ # cat /proc/meminfo
> MemTotal: 511268 kB
> MemFree: 242920 kB
> Buffers: 16580 kB
> [...]
>
> I was able to change to values below 512MB though. So the interface
> is working at least.
>
> >Lastly you should try a more recent domU kernel.
>
> today all tests above done with 3.6.6-gentoo.
> Tried 2.6.38 and 3.4.11 yesterday.
>
> Thanks for taking time,
> Christian
>
>
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |