[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] PV drivers and zero copying

Hi, Stefano!

On 07/31/2017 11:28 PM, Stefano Stabellini wrote:
On Mon, 31 Jul 2017, Oleksandr Andrushchenko wrote:
3 Sharing with page exchange (XENMEM_exchange)

This API was pointed to me by Stefano Stabellini as one of the possible ways
achieve zero copying and share physically contiguous buffers. It is used by
SWIOTLB code (xen_create_contiguous_region, [5]), but as per my understanding
this API cannot be used on ARM as of now [6].  Conclusion: not an option for
at the moment
Let me elaborate on this. The purpose of XENMEM_exchange is to exchange
a number of memory pages with an equal number of contiguous memory
pages, possibly even under 4G. The original purpose of the hypercall was
to get DMA-able memory.
this is good to know

So far, it has only been used by Dom0 on x86. Dom0 on ARM doesn't need
it because it is mapped 1:1 by default and device assignment is not
allowed without an IOMMU. However it should work on ARM too, as the
implementation is all common code in Xen.
well, according to [6]:
"Currently XENMEM_exchange is not supported on ARM because the steal_page is
left unimplemented.

However, even if steal_page is implemented, the hypercall can't work for ARM
    - Direct mapped domain is not supported
    - ARM doesn't have a M2P and therefore usage of mfn_to_gmfn is
And what I see at [7] is that it is still EOPNOTSUPP
So, yes, common code is usable for both ARM and x86, but
underlying support for ARM is till not there.
Please correct me if I am wrong here
  Also, looking at the
implementation (xen/common/memory.c:memory_exchange) it would seem that
it can be called from a DomU too (but I have never tried).
Thus, if you have a platform without IOMMU and you disabled the IOMMU
checks in Xen to assign a device to a DomU anyway, then you could use
this hypercall from DomU to get memory under 4G to be used for DMA with
this device.
There is no real device assigned to DomU, but a PV frontend

As far as I can tell XENMEM_exchange could help in the design of
zero-copy PV protocols only to address this specific use case:

- you have a frontend in DomU and a backend in Dom0
- pages shared by DomU get mapped in Dom0 and potentially used for DMA
yes, this is crucial for zero copying in my case: DMA
- the device has under 4G DMA restrictions

Normally Dom0 maps a DomU page, then at the time of using the mapped
page for DMA it checks whether it is suitable for DMA (under 4G if the
device requires so). If it is not, Dom0 uses a bounce buffer borrowed
from the swiotlb. Obviously this introduces one or two memcpys.

Instead, if DomU calls XENMEM_exchange to get memory under 4G, and
shares one of the pages with Dom0 via PV frontends, then Dom0 wouldn't
have to use a bounce buffer to do DMA to this page.

Does it make sense?
yes, it does, thank you, but [6], [7] :(

[7] https://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/arm/mm.c;h=411bab1ea9f7f14789a134056ebff9f68fd4a4c7;hb=a15516c0cf21d7ac84799f1e2e500b0bb22d2300#l1161

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.