Re: [Xen-devel] Memory fragmentation and PCI passthrough

[Pasi Kärkkäinen <pasik@xxxxxx>]

On Wed, Sep 07, 2011 at 01:36:22PM -0400, Konrad Rzeszutek Wilk wrote:
> On Tue, Sep 06, 2011 at 07:54:27PM +0200, Marek Marczykowski wrote:
> > Hello,
> > 
> > I've hit known problem with dynamic memory management - memory
> > fragmentation... This dynamic memory management basically does xl
> > mem-set to balance memory.
> > 
> > After some time of running system, xen memory is so fragmented that it
> > is impossible to start new VM with PCI device. Sometimes it crashes
> > during boot (no 64MB contiguous memory for SWIOTLB), or later - eg.
> > iwlagn cannot allocate memory for loading firmware (few allocs, each
> > bellow 100k).
> > 
> > DomU kernel cmdline: console=hvc0 iommu=soft earlyprintk=xen
> > 
> > There is two cases (I think):
> > 1. With IOMMU
> > 2. Without IOMMU
> > I've tried only the second one.
> > 
> > Is there any known solution for this problem?
> > Some ideas:
> > 1. With IOMMU pass iommu=pv to Xen. AFAIU domU will not need iommu=soft
> > parameter then, right? Will it work then with fragmented memory?
> 
> It will still need it. Otherwise the Xen SWIOTLB won't be used.
> 

Btw I think Xen hypervisor "iommu=pv" is a deprecated option.. 
"iommu=soft" for the domU kernel is a valid option though :)


-- Pasi


> But you can limit the amount of memory for the SWIOTLB, say by
> doing 'swiotlb=2048'
> 
> > 
> > 2. Force somehow on xen/libxl to allocate memory (for domU) in chunks
> > of, say 4MB, to not fragment it so badly. Is it doable?
> > 
> > In tmem documentation is also described some workaround for this:
> > reserve some memory region for allocations with 0<order<=9. But SWIOTLB
> > tries to allocate 64MB, which much bigger than 2MB... Is it really
> > needed to allocate such big region of contiguous memory in one piece?
> 
> Nope. It only uses that pool for 32-bit devices too - so there is not
> really a big need for it.
> 
> 
> > 
> > -- 
> > Pozdrawiam / Best Regards,
> > Marek Marczykowski
> > Invisible Things Lab
> > 
> 
> 
> 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel