Re: [Xen-devel] [PATCH 2/2] docs: expand persistent grants protocol

[Roger Pau Monnà <roger.pau@xxxxxxxxxx>]

On 29/11/12 11:49, Ian Campbell wrote:
> On Tue, 2012-11-27 at 10:03 +0000, Roger Pau Monne wrote:
>> Signed-off-by: Roger Pau Monnà <roger.pau@xxxxxxxxxx>
>> ---
>>  xen/include/public/io/blkif.h |   24 +++++++++++++++++++++---
>>  1 files changed, 21 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/include/public/io/blkif.h b/xen/include/public/io/blkif.h
>> index db9c379..5a4b9ae 100644
>> --- a/xen/include/public/io/blkif.h
>> +++ b/xen/include/public/io/blkif.h
>> @@ -137,7 +137,15 @@
>>   *      can map persistently depends on the implementation, but ideally it
>>   *      should be RING_SIZE * BLKIF_MAX_SEGMENTS_PER_REQUEST. Using this
>>   *      feature the backend doesn't need to unmap each grant, preventing
>> - *      costly TLB flushes.
>> + *      costly TLB flushes. The backend driver should only map grants
>> + *      persistently if the frontend supports it. If a backend driver 
>> chooses
>> + *      to use the persistent protocol when the frontend doesn't support it,
>> + *      it will probably hit the maximum number of persistently mapped 
>> grants
>> + *      (due to the fact that the frontend won't be reusing the same 
>> grants),
>> + *      and fall back to non-persistent mode. Backend implementations may
>> + *      shrink or expand the number of persistently mapped grants without
>> + *      notifying the frontend depending on memory constraints (this might
>> + *      cause a performance degradation).
> 
> Is there a recommended/required reuse strategy on either end which
> minimises this? You don't want to be in a situation where the backend's
> "cache" is full of non-persistent single-shot mappings but the frontend
> is now reusing a good set of persistent pages, which are getting
> repeatedly mapped/unmapped because the cache is full...

Since the only backend implementation is the Linux one, and we set the
maximum number of persistenly mapped grants to RING_SIZE *
BLKIF_MAX_SEGMENTS_PER_REQUEST, we don't have any kind of support for
shrinking/expanding. If a frontend is using more than this number of
grants it is considered that the frontend is broken/hostile.

If in the future some kind of shrinking/expanding is implemented, I
guess the natural way to do it would be to add a counter to keep track
of how many times a grant is used, and try to maintain the grants most
commonly used mapped.

On the blkfront side, we are using a LIFO strategy to store persistently
mapped grants, so we are trying to always reuse the same subset of
persistently mapped grants when blkfront is not under heavy I/O load.

> 
>>   *
>>   *----------------------- Request Transport Parameters 
>> ------------------------
>>   *
>> @@ -258,11 +266,17 @@
>>   * feature-persistent
>>   *      Values:         0/1 (boolean)
>>   *      Default Value:  0
>> - *      Notes: 7, 8
>> + *      Notes: 7, 8, 9
>>   *
>>   *      A value of "1" indicates that the frontend will reuse the same 
>> grants
>>   *      for all transactions, allowing the backend to map them with write
>> - *      access (even when it should be read-only).
>> + *      access (even when it should be read-only). If the frontend hits the
>> + *      maximum number of allowed persistenlty mapped grants, it can 
>> fallback
> 
>                                     persistently
> 
>> + *      to non persistent mode. This will cause a performance degradation,
>> + *      since the the backend driver will still try to map those grants
>> + *      persistently. Since the persistent grants protocol is compatible 
>> with
>> + *      the previous protocol, a frontend driver can choose to work in
>> + *      persistent mode even when the backend doesn't support it.
>>   *
>>   *------------------------- Virtual Device Properties 
>> -------------------------
>>   *
>> @@ -308,6 +322,10 @@
>>   * (8) The frontend driver has to allow the backend driver to map all grants
>>   *     with write access, even when they should be mapped read-only, since
>>   *     further requests may reuse this grants and require write permissions.
>> + * (9) Linux implementation doesn't have a limit on the maximum number of
>> + *     grants that can be persisntly mapped in the frontend driver, but
> 
>                             persistently
> 
>> + *     due to the frontent driver implementation it should never be bigger
>> + *     than RING_SIZE * BLKIF_MAX_SEGMENTS_PER_REQUEST.
>>   */
>>  
>>  /*
> 
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel