Re: [Xen-devel] [Hackathon minutes] PV block improvements

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Thu, 27 Jun 2013, Roger Pau Monnà wrote:
> On 21/06/13 20:07, Matt Wilson wrote:
> > On Fri, Jun 21, 2013 at 07:10:59PM +0200, Roger Pau Monnà wrote:
> >> Hello,
> >>
> >> While working on further block improvements I've found an issue with
> >> persistent grants in blkfront.
> >>
> >> Persistent grants basically allocate grants and then they are never
> >> released, so both blkfront and blkback keep using the same memory pages
> >> for all the transactions.
> >>
> >> This is not a problem in blkback, because we can dynamically choose how
> >> many grants we want to map. On the other hand, blkfront cannot remove
> >> the access to those grants at any point, because blkfront doesn't know
> >> if blkback has this grants mapped persistently or not.
> >>
> >> So if for example we start expanding the number of segments in indirect
> >> requests, to a value like 512 segments per requests, blkfront will
> >> probably try to persistently map 512*32+512 = 16896 grants per device,
> >> that's much more grants that the current default, which is 32*256 = 8192
> >> (if using grant tables v2). This can cause serious problems to other
> >> interfaces inside the DomU, since blkfront basically starts hoarding all
> >> possible grants, leaving other interfaces completely locked.
> > 
> > Yikes.
> > 
> >> I've been thinking about different ways to solve this, but so far I
> >> haven't been able to found a nice solution:
> >>
> >> 1. Limit the number of persistent grants a blkfront instance can use,
> >> let's say that only the first X used grants will be persistently mapped
> >> by both blkfront and blkback, and if more grants are needed the previous
> >> map/unmap will be used.
> > 
> > I'm not thrilled with this option. It would likely introduce some
> > significant performance variability, wouldn't it?
> > 
> >> 2. Switch to grant copy in blkback, and get rid of persistent grants (I
> >> have not benchmarked this solution, but I'm quite sure it will involve a
> >> performance regression, specially when scaling to a high number of 
> >> domains).
> > 
> > Why do you think so?
> 
> I've hacked a prototype blkback using grant_copy instead of persistent
> grants, and removed the persistent grants support in blkfront and indeed
> the performance of grant_copy is lower than persistent grants, and it
> seems to scale much worse. I've run several fio read/write benchmarks,
> using 512 segments per request on a ramdisk, and the output is the
> following:
> 
> http://xenbits.xen.org/people/royger/grant_copy/

Very impressive. We should consider doing the same experiment with
netfront/netback at some point.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel