Re: [Xen-devel] [PATCH 1/1] xen/blkfront: always allocate grants first from per-queue persistent grants

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, Jun 28, 2017 at 12:43:03PM +0800, Dongli Zhang wrote:
> This patch partially reverts 3df0e50 ("xen/blkfront: pseudo support for
> multi hardware queues/rings"). The xen-blkfront queue/ring might hang due
> to grants allocation failure in the situation when gnttab_free_head is
> almost empty while many persistent grants are reserved for this queue/ring.
> 
> As persistent grants management was per-queue since 73716df ("xen/blkfront:
> make persistent grants pool per-queue"), we should always allocate from
> persistent grants first.
> 
> Signed-off-by: Dongli Zhang <dongli.zhang@xxxxxxxxxx>
> ---
>  drivers/block/xen-blkfront.c | 17 ++++++++++-------
>  1 file changed, 10 insertions(+), 7 deletions(-)
> 
> diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
> index 3945963..d2b759f 100644
> --- a/drivers/block/xen-blkfront.c
> +++ b/drivers/block/xen-blkfront.c
> @@ -713,6 +713,7 @@ static int blkif_queue_rw_req(struct request *req, struct 
> blkfront_ring_info *ri
>        * existing persistent grants, or if we have to get new grants,
>        * as there are not sufficiently many free.
>        */
> +     bool new_persistent_gnts = false;
>       struct scatterlist *sg;
>       int num_sg, max_grefs, num_grant;
>  
> @@ -724,12 +725,13 @@ static int blkif_queue_rw_req(struct request *req, 
> struct blkfront_ring_info *ri
>                */
>               max_grefs += INDIRECT_GREFS(max_grefs);
>  
> -     /*
> -      * We have to reserve 'max_grefs' grants because persistent
> -      * grants are shared by all rings.
> -      */
> -     if (max_grefs > 0)
> -             if (gnttab_alloc_grant_references(max_grefs, &setup.gref_head) 
> < 0) {
> +     /* Check if we have enough persistent grants to allocate a requests */
> +     if (rinfo->persistent_gnts_c < max_grefs) {
> +             new_persistent_gnts = true;
> +
> +             if (gnttab_alloc_grant_references(
> +                 max_grefs - rinfo->persistent_gnts_c,
> +                 &setup.gref_head) < 0) {
>                       gnttab_request_free_callback(
>                               &rinfo->callback,
>                               blkif_restart_queue_callback,

AFAICT you should also change the call to gnttab_request_free_callback
to request for max_grefs - rinfo->persistent_gnts_c. In any case the
number of persistent grants is not going to decrease now, because the
buffer is per-queue, so the only thing that can happen is that
requests complete and the number of persistent grants increase.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel