Re: [PATCH v2 10/11] xen/arm: introduce phys/dma translations in xen_dma_sync_for_*

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Mon, 8 Jun 2020, Christoph Hellwig wrote:
> On Wed, Jun 03, 2020 at 03:22:46PM -0700, Stefano Stabellini wrote:
> > From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > 
> > xen_dma_sync_for_cpu, xen_dma_sync_for_device, xen_arch_need_swiotlb are
> > getting called passing dma addresses. On some platforms dma addresses
> > could be different from physical addresses. Before doing any operations
> > on these addresses we need to convert them back to physical addresses
> > using dma_to_phys.
> > 
> > Add dma_to_phys calls to xen_dma_sync_for_cpu, xen_dma_sync_for_device,
> > and xen_arch_need_swiotlb.
> > 
> > dma_cache_maint is fixed by the next patch.
> 
> The calling conventions because really weird now because
> xen_dma_sync_for_{device,cpu} already get both a phys_addr_t and
> a dma_addr_t.  
> 
> > 
> > -   if (pfn_valid(PFN_DOWN(handle)))
> > +   if (pfn_valid(PFN_DOWN(dma_to_phys(dev, handle))))
> 
> But here we translate the dma address to a phys addr
> 
> >             arch_sync_dma_for_cpu(paddr, size, dir);
> 
> While this still uses the passed in paddr.  I think the uses of
> addresses in this code really needs a major rethink.


Yeah, the pfn_valid check is a bit weird by definition because we are
using it to understand whether the address belong to us or to another
VM. To do the pfn_valid check we need to translate the dma address into
something the CPU understands, hence, the dma_to_phys call.

Why can't we use the already-provided paddr? Because paddr has been
translated twice:
1) from dma to maybe-foreign phys address (could be ours, or another VM)
2) from maybe-foreign address to local (using our local mapping of the foreign 
page)

In fact, it would be clearer if we had all three addresses as parameters
of xen_dma_sync_for_cpu: the dma address, the maybe-foreign physical
address (we tend to call it xenbus address, baddr), the local physical
address. Something like:


void xen_dma_sync_for_cpu(struct device *dev, dma_addr_t handle,
                          phys_addr_t baddr, phys_addr_t paddr, size_t size,
                          enum dma_data_direction dir)
{
        if (pfn_valid(baddr))
                arch_sync_dma_for_cpu(paddr, size, dir);
        else if (dir != DMA_TO_DEVICE)
                dma_cache_maint(dev, handle, size, GNTTAB_CACHE_INVAL);
}