|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-users] Enabling SMMU for a bare metal guest
On Wed, Jun 22, 2016 at 12:11:53PM -0400, Jarvis Roach wrote:
> Hi Julien,
>
> Thanks for taking a look, and please excuse any faux paus of mailing list
> etiquette I am about to commit...
>
>
> On Wed, Jun 22, 2016 at 10:43 AM, Julien Grall <julien.grall@xxxxxxx> wrote:
>
> > Hello Jarvis,
> >
> > I have CCed Edgar who did the bring of Xen on Xilinx Zinqmp
>
>
> >
> > On 21/06/16 15:40, Jarvis Roach wrote:
> >
> >> I've been working on passing an ethernet controller through to a bare
> >> metal guest running on an Xilinx Z US+ MPSoC platform. I got it working,
> >> but only by using the system's physical address (PA) for DMA buffers and
> >> descriptors instead of using the guest's intermediate physical address
> >> (IPA), aka guest phyiscal address.
> >>
> >> I am aware of Julien's presentation on passthrough, and I am successful
> >> in passing the ethernet device through to a Linux guest, but when I
> >> follow the same steps for my bare metal guest, the IPA->PA translation
> >> doesn't seem to be happening. I would expect that if the SMMU was active
> >> for that ethernet controller then a) using PA for DMA buffers and
> >> descriptors should result in a translation fault, and b) using IPA for
> >> the same would work, neither of which is true.
> >>
> >
> > Can you provide the guest configuration you are using?
>
>
> I'll start with the revelant snippets. Please let me know if you'd like to
> see the entire files.
>
> Relevant xen.dts:
>
> smmu@fd800000 {
> compatible = "arm,mmu-500";
> reg = <0x0 0xfd800000 0x0 0x20000>;
> #global-interrupts = <0x1>;
> interrupt-parent = <0x1>;
> interrupts = <0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b
> 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0
> 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4 0x0 0x9b 0x4
> 0x0 0x9b 0x4>;
> mmu-masters = <&ser1 0x876 &ttc1 0x877 &gem3 0x878>;
Hi Jarvis,
These don't look correct.
These are the GEM master-ids:
&gem0 0x874 &gem1 0x875 &gem2 0x876 &gem3 0x877
The serial ports and ttc timers are not DMA capable and shouldn't be listed
there.
Also these master-ids are tied to a device in HW, it's not possible to
edit/change these in SW or DTS at choice.
Here's a list the DTS in linux:
mmu-masters = < &gem0 0x874
&gem1 0x875
&gem2 0x876
&gem3 0x877
&usb0 0x860
&usb1 0x861
&qspi 0x873
&lpd_dma_chan1 0x868
&lpd_dma_chan2 0x869
&lpd_dma_chan3 0x86a
&lpd_dma_chan4 0x86b
&lpd_dma_chan5 0x86c
&lpd_dma_chan6 0x86d
&lpd_dma_chan7 0x86e
&lpd_dma_chan8 0x86f
&fpd_dma_chan1 0x14e8
&fpd_dma_chan2 0x14e9
&fpd_dma_chan3 0x14ea
&fpd_dma_chan4 0x14eb
&fpd_dma_chan5 0x14ec
&fpd_dma_chan6 0x14ed
&fpd_dma_chan7 0x14ee
&fpd_dma_chan8 0x14ef
&sdhci0 0x870
&sdhci1 0x871
&nand0 0x872>;
Best regards,
Edgar
>
> };
> .
> .
> .
> gem3:ethernet@ff0e0000 {
> #stream-id-cells = <0x1>;
> compatible = "cdns,gem";
> status = "disabled";
> interrupt-parent = <0x1>;
> interrupts = <0x0 0x3f 0x4 0x0 0x3f 0x4>;
> reg = <0x0 0xff0e0000 0x1000>;
> clock-names = "pclk", "hclk", "tx_clk";
> #address-cells = <0x1>;
> #size-cells = <0x0>;
> jumbo-max-len = <0x2800>;
> jumbo-supported;
> clocks = <0x3 0x3 0x3>;
> phy-mode = "rgmii-id";
> xlnx,ptp-enet-clock = <0x0>;
> local-mac-address = [00 0a 35 00 22 01];
> phy-handle = <0x4>;
> xen,passthrough = <0x1>;
> linux,phandle = <0x12>;
> phandle = <0x12>;
>
> phy@21 {
> reg = <0x15>;
> ti,rx-internal-delay = <0x8>;
> ti,tx-internal-delay = <0xa>;
> ti,fifo-depth = <0x1>;
> linux,phandle = <0x4>;
> phandle = <0x4>;
> };
> };
>
>
>
> Guest configuration:
>
> name = "bare1"
> kernel = "bare.img"
> memory = 8
> vcpus = 1
> cpus = [1]
>
> dtdev = [ "/amba/serial@ff010000", "/amba/ethernet@ff0e0000",
> "/amba/timer@ff110000" ]
> #device_tree = "partial.dtb"
> irqs = [ 54, 95, 68, 69, 70 ]
> iomem = [ "0xff010,1", "0xff0e0,1", "0xff110,1" ]
>
>
> Is the partial.dtb only for the guest's benefit? I had assumed that was the
> case, and since my guest didn't speak device-tree I left it out initially,
> but I did go back and try it with a partial just in case there was some
> addtional configuration triggered by the presence of it in the .cfg file
> without any apparent change in behavior.
>
>
> >> It seems to me that there is something extra is being done by, or for,
> >> the Linux guest that isn't done by, or for, my bare metal guest that
> >> enables the SMMU for that ethernet controller. I haven't been able to
> >> figure what that something extra is by searching the internet, reading
> >> through the available documentation, or digging into the code. Any help
> >> (better search terms, useful URLs, detailed description of the what's
> >> supposed to happen for guests generically or what actually happens for a
> >> Linux guest, a working bare metal example, etc) would be appreciated.
> >>
> >
> > The SMMU is configured with Stage-2 page table and not exposed to the
> > guest. I am not aware of any configuration required in the guest, except
> > having a working network driver.
>
>
> >
> "Working" can be a tricky term to nail down. The driver "works", as in I
> can telnet/ping/nc to my target if I pass in PAs instead of IPAs to the
> DMA, but maybe "working" involves some step that the PV-aware Linux guest
> is doing that I'm not; that something occurs as a by-product of Linux guest
> behavior, like maybe a page grant that occurs as part of a DMA buffer
> allocation, that completes the SMMU configuration, but I haven't been able
> to find anything like yet. If there's any documentation on what guests
> should be doing in this regard I'd welcome a reference to it.
>
>
>
> > Regards,
> >
> > --
> > Julien Grall
> >
>
> Thanks again for your time and attention.
>
> -J
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |