Re: [Xen-devel] blkback global resources

[Wei Liu <wei.liu2@xxxxxxxxxx>]

On Mon, 2012-03-26 at 17:20 +0100, Ian Campbell wrote:
> On Mon, 2012-03-26 at 16:56 +0100, Jan Beulich wrote:
> > All the resources allocated based on xen_blkif_reqs are global in
> > blkback. While (without having measured anything) I think that this
> > is bad from a QoS perspective (not the least implied from a warning
> > issued by Citrix'es multi-page-ring patches:
> > 
> >     if (blkif_reqs < BLK_RING_SIZE(order))
> >             printk(KERN_WAfdfdRNING "WARNING: "
> >                    "I/O request space (%d reqs) < ring order %ld, "
> >                    "consider increasing %s.reqs to >= %ld.",
> >                    blkif_reqs, order, KBUILD_MODNAME,
> >                    roundup_pow_of_two(BLK_RING_SIZE(order)));
> > 
> > indicating that this _is_ a bottleneck), I'm otoh hesitant to convert
> > this to per-instance allocations, as the amount of memory taken
> > away from Dom0 for this may be not insignificant when there are
> > many devices.
> > 

What's your main concern on QoS? Lock contention? Starvation? Or any
other things?

You are right here, we should be careful on how much memory blkback may
consume.

> > Does anyone have an opinion here, in particular regarding the
> > original authors' decision to make this global vs. the apparently
> > made observation (by Daniel Stodden, the author of said patch,
> > who I don't have any current email of to ask directly), but also
> > in the context of multi-page rings, the purpose of which is to
> > allow for larger amounts of in-flight I/O?
> 

I don't know the actual bottleneck for blkback because I never play with
it. But for multi-page ring netback / netfront, global pool (in the
sense of starvation) may not become bottleneck if there are very limited
VIFs running. What I observe is that to achieve 3.8Gb/s throughput
between two VIFs connected throughput internal bridge, it only consumes
~80 pages in the pool.


Wei.

> Not really much to say other than we (well, mostly Wei Liu) have a
> similar issue with netback too. That used to have a global pool, then a
> pool-per-worker thread. When Wei added thread-per-vif support he solved
> this by adding a "page pool" which handles allocations. Possibly this
> could grow some sort of fairness etc nobs and be shared with blkback?
> 
> Ian.
> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel