Re: [Xen-devel] [PATCH v02 1/7] arm: introduce remoteprocessor iommu module

[Andrii Tseglytskyi <andrii.tseglytskyi@xxxxxxxxxxxxxxx>]

On Wed, Jul 16, 2014 at 6:29 PM, Ian Campbell <Ian.Campbell@xxxxxxxxxx> wrote:
> On Thu, 2014-06-26 at 14:07 +0300, Andrii Tseglytskyi wrote:
>> This is a fisrst patch from patch series which was
>> developed to handle remote (external) processors
>> memory management units. Remote processors are
>> typically used for graphic rendering (GPUs) and
>> high quality video decoding (IPUs). They are typically
>> installed on such multimedia SoCs as OMAP4 / OMAP5.
>>
>> As soon as remoteprocessor MMU typically works with
>> pagetables filled by physical addresses, which are
>> allocated by domU kernel, it is almost impossible to
>> use them under Xen, intermediate physical addresses
>> allocated by kernel, need to be translated to machine
>> addresses.
>>
>> This patch introduces a simple framework to perform
>> pfn -> mfn translation for external MMUs.
>> It introduces basic data structures and algorithms
>> needed for translation.
>>
>> Typically, when MMU is configured, some it registers
>> are updated by new values. Introduced frameworks
>> uses traps as starting point of remoteproc MMUs
>> pagetables translation.
>
> Just to clarify the model:
>
> This is intended as a framework to enable Xen to provide an SMMU to the
> guest. Is the intention that this SMMU be an idealised PV thing or is it
> intending to emulate a real SMMU (and therefore the guests existing
> drivers)? I think this is the second one.
>

For sure it is a second one.

> And the intention here is to let the guest manipulate the page tables
> directly and shadow them from IPA->MA as opposed to e.g. exposing the
> guest's complete IPA->MA mapping to the remoteproc.
>

Right.

> I suppose either way my main concern is going to be the potential for
> proliferation of backends (and perhaps frontends) for all sorts of
> different H/W MMUs. I suppose that is unavoidable though.
>

Right. The only comment - code is separated between files.
*omap_iommu.c* contains H/W specific definitions for existing IPU and GPU
*remoteproc_iommu.c* is designed to contain generic code. In ideal
world any platform with any remoteprocs, which need pagetables
translation will add it's own file with specific HW definitions and
hooks to remoteproc_iommu.c. Of course this is true only for "ideal
world". I never tried to run this code on other platforms than OMAP5,
and I would expect that generic part may need adjustments for other
platforms.


> WRT the shadowing, I don't see the bit which causes the guest pages
> which it is using as PT page for the remoteproc to get marked R/O to
> force a trap. Perhaps that comes later.
>

Hmm. thank you for this comment. To make this framework working I
unmap MMU device memory pages in domain config file. On my setup IPU
and SGX belongs to Android domU kernel and all their HW such as
interrupts and iomem are forwarded as is in config file.
Due to this the only I need  - is to unmap their MMU device memory. Do
you mean that it can be done in a different way in hypervisor code?

> Do these remoteproc page tables get frequently updated at run time or
> are they mostly just build once and forget?
>

IPU - boot time and that's it. It allocates a big (about 256 Mb) chunk
of memory. In my kernel, driver which handles remoteproc is moved to
kernel module which starts after graphic - therefore user desn't see
*any* delays on boot and any performance drop.

GPU - allocates every time when rendering is performed. Also it
creates new standalone MMU pagetable on each new screen. And this was
the most painful case - to handle dynamic page allocations /
translations without any performance drop. But looks like an attempt
was successful - framework does not change complete pagetabe every
time. It handles difference between previous pagetable and current
kernel pagetable and adds / deletes only that pages, which were
allocated / deleted. Of course, taking in account what Stefano saying
about Xen mfns mapping changes - it will need modifications.

Regards,
Andrii

> Ian.
>



-- 

Andrii Tseglytskyi | Embedded Dev
GlobalLogic
www.globallogic.com

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel