Re: Help: Information required for AXI DMA passthrough for PL masters in ZCU102 dev kit

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>]

>From the gpa address, it looks like FreeRTOS is trying to access the TTC
timer? If you look at device tree, the corresponding node is
timer@ff110000.

If FreeRTOS is configured to access the TTC timer, then you need to
assign it to it. Did you assign the TTC timer to the FreeRTOS domU by
any chance?

You can do that by adding:

irqs = [ 68, 69, 70 ]
iomem = [ "0xff110,1" ]


On Fri, 28 May 2021, Prem Harikrishnan wrote:
> Hi Stefano,
> Thanks for your reply. I am pretty sure I followed the steps you mentioned 
> below to run the FreeRTOS hello world example. I already added
> printfs and i can see XEN cannot start the scheduler and i get the following 
> out put on XEN.
> 
> Parsing config from freertos.cfg (XEN) Dom2 colors: [ 8 ]
> (XEN) memory.c:255:d0v0 Could not allocate order=9 extent: id=2 memflags=0xc0 
> (0 of 64)
> (XEN) memory_map:add: dom2 gfn=ff010 mfn=ff010 nr=1 cache=0
> (XEN) sched_null.c:344: 1 <-- d2v0
> (XEN) traps.c:1973:d2v0 HSR=0x93820007 pc=0x00000040012b08 gva=0xff11000c 
> gpa=0x000000ff11000c
> 
> 
> I also see that according to this article
> https://forums.xilinx.com/t5/Adaptable-Advantage-Blog/Partner-FreeRTOS-as-a-Xen-Virtual-Machine/ba-p/947030
> [52662iFAD1733707609204?v=v2]
> Partner: FreeRTOS as a Xen Virtual Machine - Community Forums
> Contributed by Jeff Kubascik, Embedded Systems Engineer at DornerWorks What 
> is FreeRTOS? The FreeRTOS kernel is a market leading real-time
> operating system for embedded systems. As embedded applications grow in 
> complexity, there is a growing need for a lightweight kernel
> capable of task scheduling...
> forums.xilinx.com
> FreeRTOS does not run out of the box for XEN, although the article is 2 years 
> old, I am not sure if there are any updates on this after
> that. Can you please confirm?
> 
> Thanks and Best Regards,
> Prem
> 
> ___________________________________________________________________________________________________________________________________________
> From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> Sent: Friday, May 28, 2021 02:51
> To: Prem Harikrishnan <prem.harikrishnan@xxxxxxxxxxxxxx>
> Cc: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>; 
> xen-users@xxxxxxxxxxxxxxxxxxxx <xen-users@xxxxxxxxxxxxxxxxxxxx>; 
> JOVITAC@xxxxxxxxxx
> <JOVITAC@xxxxxxxxxx>; MUBINUSM@xxxxxxxxxx <MUBINUSM@xxxxxxxxxx>
> Subject: Re: Help: Information required for AXI DMA passthrough for PL 
> masters in ZCU102 dev kit  
> +Mubin
> 
> 
> Hi Prem,
> 
> Did you make sure to build FreeRTOS as "hypervisor guest"? I think
> FreeRTOS is built to run at EL3 by default so if that is not changed
> there are errors when run on any hypervisor. It is possible to build
> FreeRTOS to run on Xen but it requires a couple of manual steps:
> 
> - source settings64.sh from Vitis builds
> - xsct
> - set WORKSPACE .
> - setws -switch $WORKSPACE
> - app create -name freertos-hello-world -template {FreeRTOS Hello World} 
> -proc psu_cortexa53_0 -hw {xsa path} -os freertos10_xilinx
> - bsp config stdin psu_uart_1
> - bsp config stdout psu_uart_1
> - bsp config hypervisor_guest true
> - Go to application source path and modify the linker script start address to 
> 0x40000000,
> - app build -name freertos-hello-world
> - Run command to generate BIN file from executable "aarch64-linux-gnu-objcopy 
> -O binary --gap-fill 0  <elf path> BOOT.BIN"
> 
> The resulting BOOT.BIN can be started a Xen DomU guest. Mubin, CC'ed,
> might be able to provide additional info.
> 
> 
> 
> On Thu, 27 May 2021, Prem Harikrishnan wrote:
> > Hi Stefano,
> > Sorry to bother you again.
> > I am currently trying to run my custom IP which sends periodic dma 
> > interrupts to the PS with cache coloring enabled. I have now compiled
> my
> > guest running with FreeRTOS and it generates a trap.
> > Petalinux and VITIS IDE Version is 2020.2.
> >
> > Please find the log attached. After the trap occurs, for every interrupt 
> > XEN prints (XEN) smmu: /smmu@fd800000: Unhandled context fault:
> > fsr=0x402, iova=0x41000000, fsynr=0x12, cb=1
> >
> > root@pilbara:/etc/xen# xl create -c apu_freertos.cfg Parsing config from 
> > apu_freertos.cfg
> > (XEN) Dom1 colors: [ 8 ]
> > (XEN) memory.c:255:d0v0 Could not allocate order=9 extent: id=1 
> > memflags=0xc0 (0 of 64)
> > (XEN) memory_map:add: dom1 gfn=ff010 mfn=ff010 nr=1 cache=0
> > (XEN) memory_map:add: dom1 gfn=a0010 mfn=a0010 nr=1 cache=0
> > (XEN) smmu: /smmu@fd800000: d1: p2maddr 0x000000087f554000
> > (XEN) sched_null.c:344: 1 <-- d1v0
> > (XEN) traps.c:1973:d1v0 HSR=0x93820007 pc=0x00000040013188 gva=0xff11000c 
> > gpa=0x000000ff11000c
> > (XEN) smmu: /smmu@fd800000: Unhandled context fault: fsr=0x402, 
> > iova=0x41000000, fsynr=0x12, cb=1
> > (XEN) smmu: /smmu@fd800000: Unhandled context fault: fsr=0x402, 
> > iova=0x41000000, fsynr=0x12, cb=1
> > (XEN) smmu: /smmu@fd800000: Unhandled context fault: fsr=0x402, 
> > iova=0x41000000, fsynr=0x12, cb=1
> > (XEN) smmu: /smmu@fd800000: Unhandled context fault: fsr=0x402, 
> > iova=0x41000000, fsynr=0x12, cb=1
> >
> > The same application works as expected without FreeRTOS, i.e as a baremetal 
> > guest. I look forward for your comment on this.
> >
> > Although i see performance degradation, normally i can measure delta time 
> > for each interrupt configured to 1ms (without XEN), with XEN
> for
> > the same measurement shows 16 ms for each interval (but the performance is 
> > not a priority for now).
> > Thanks and Best Regards,
> > Prem
> >
> >__________________________________________________________________________________________________________________________________________
> _
> > From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > Sent: Tuesday, May 18, 2021 20:09
> > To: Prem Harikrishnan <prem.harikrishnan@xxxxxxxxxxxxxx>
> > Cc: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>; 
> > xen-users@xxxxxxxxxxxxxxxxxxxx <xen-users@xxxxxxxxxxxxxxxxxxxx>;
> JOVITAC@xxxxxxxxxx
> > <JOVITAC@xxxxxxxxxx>
> > Subject: Re: Help: Information required for AXI DMA passthrough for PL 
> > masters in ZCU102 dev kit  
> > Great to hear! Thank you for reporting back.
> >
> >
> > On Tue, 18 May 2021, Prem Harikrishnan wrote:
> > > Dear Stefano,
> > > Thank you very much for your guidance and support. I finally fixed the 
> > > issue. I made an error in base memory address like you pointed
> > out.
> > > I actually modified the base address correctly to 0x40000000 but the 
> > > default C code had an additional offset of 0x1100000 , and the
> DomU
> > > memory allocated was only 8MB. I either had to change the offset or 
> > > allocate more memory to the DomU guest which i missed until you
> > pointed
> > > out. Additionally i had to set to AxPROT bits to 0x2 in Vivado and the 
> > > SMID required is 0xE80.
> > >
> > > Thanks again i was able get my custom IP also working as an AXI master 
> > > from PL.
> > >
> > > Best Regards,
> > > Prem
> > >
> >>_________________________________________________________________________________________________________________________________________
> _
> > _
> > > From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > > Sent: Tuesday, May 18, 2021 02:35
> > > To: Prem Harikrishnan <prem.harikrishnan@xxxxxxxxxxxxxx>
> > > Cc: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>; Brian Woods 
> > > <brian.woods@xxxxxxxxxx>; Julien Grall <julien@xxxxxxx>; Stefano
> > > Stabellini <sstabellini@xxxxxxxxxx>; xen-users@xxxxxxxxxxxxxxxxxxxx 
> > > <xen-users@xxxxxxxxxxxxxxxxxxxx>; fnuv@xxxxxxxxxx
> <fnuv@xxxxxxxxxx>;
> > > edgari@xxxxxxxxxx <edgari@xxxxxxxxxx>; bwoods288@xxxxxxxxx 
> > > <bwoods288@xxxxxxxxx>; Renato Costa Amorim <renato.amorim@xxxxxxxxxxxxxx>;
> > > JOVITAC@xxxxxxxxxx <JOVITAC@xxxxxxxxxx>
> > > Subject: Re: Help: Information required for AXI DMA passthrough for PL 
> > > masters in ZCU102 dev kit  
> > > Your configuration looks correct as far as I can tell.
> > >
> > > You need dtdev because it triggers the SMMU configuration for your PL
> > > block. You don't need device_tree because your baremetal application is
> > > not going to read it, but it is not going to cause any issues either.
> > >
> > > When you say "the execution does not complete the loop at line 244", it
> > > means that you see the execution entering the loop and then never
> > > leaving? Like an infinite loop?
> > >
> > > If so, then I imagine the issue could be that the application is
> > > crashing when trying to access TxBufferPtr, which is pointing to
> > > TX_BUFFER_BASE, which is:
> > >
> > >
> > > #ifndef DDR_BASE_ADDR
> > > #warning CHECK FOR THE VALID DDR ADDRESS IN XPARAMETERS.H, \
> > >                  DEFAULT SET TO 0x01000000
> > > #define MEM_BASE_ADDR           0x01000000
> > > #else
> > > #define MEM_BASE_ADDR           (DDR_BASE_ADDR + 0x1000000)
> > > #endif
> > >
> > > #define TX_BUFFER_BASE          (MEM_BASE_ADDR + 0x00100000)
> > >
> > >
> > >
> > > Can you double check that TX_BUFFER_BASE is pointing to a correct
> > > address? Memory starts at 0x40000000 in a Xen DomU.
> > >
> > > Maybe MEM_BASE_ADDR/DDR_BASE_ADDR is not set correctly?
> > >
> > >
> > > On Mon, 17 May 2021, Prem Harikrishnan wrote:
> > > > Hi Stefano,
> > > > Thanks again for your reply and your suggestion.  I did try to replace 
> > > > the Xil_DCacheFlushRange with Xil_DCacheInvalidateRange(works
> > only
> > > > in EL3), I added printfs to debug, i can see that the execution does 
> > > > not complete the loop at line 244
> > > > https://github.com/Xilinx/embeddedsw/blob/master/XilinxProcessorIPLib/drivers/axidma/examples/xaxidma_example_simple_poll.c
> > > >   and
> > > > Xil_DCacheFlushRange is not reached. Please find attached my xen 
> > > > bootlog.txt.
> > > > I am still unable to find out why??
> > > >
> > > > Can you please atleast verify that my xen.dtsi, and pl.dtb is correct? 
> > > > and do i need dtdev and device_tree in my guest config file
> when
> > > it
> > > > is already added to xen.dtsi?
> > > >
> > > > Best Regards,
> > > > Prem
> > > >
> > > >
> >>>________________________________________________________________________________________________________________________________________
> _
> > _
> > > _
> > > > From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > > > Sent: Saturday, May 15, 2021 00:25
> > > > To: Prem Harikrishnan <prem.harikrishnan@xxxxxxxxxxxxxx>
> > > > Cc: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>; Brian Woods 
> > > > <brian.woods@xxxxxxxxxx>; Julien Grall <julien@xxxxxxx>; Stefano
> > > > Stabellini <sstabellini@xxxxxxxxxx>; xen-users@xxxxxxxxxxxxxxxxxxxx 
> > > > <xen-users@xxxxxxxxxxxxxxxxxxxx>; fnuv@xxxxxxxxxx
> > <fnuv@xxxxxxxxxx>;
> > > > edgari@xxxxxxxxxx <edgari@xxxxxxxxxx>; bwoods288@xxxxxxxxx 
> > > > <bwoods288@xxxxxxxxx>; Renato Costa Amorim 
> > > > <renato.amorim@xxxxxxxxxxxxxx>;
> > > > JOVITAC@xxxxxxxxxx <JOVITAC@xxxxxxxxxx>
> > > > Subject: Re: Help: Information required for AXI DMA passthrough for PL 
> > > > masters in ZCU102 dev kit  
> > > > Hi Prem,
> > > >
> > > > Some older versions of Xil_DCacheFlushRange were implemented using
> > > > deprecated set/way instructions that are known to cause problems in
> > > > virtualized environments.
> > > >
> > > > More recent implementations use the "dc civac" intruction that shouldn't
> > > > have any issues.
> > > >
> > > > Can you double-check that you are calling the 64-bit version implemented
> > > > by CIVAC?
> > > >
> >>>https://github.com/Xilinx/embeddedsw/blob/875dcc7a4dca47ccb18eda0182f3577e598f8917/lib/bsp/standalone/src/arm/ARMv8/64bit/xil_cache.c#L4
> 1
> > 4
> > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, 14 May 2021, Prem Harikrishnan wrote:
> > > > > Hi Stefano, 
> > > > > Thanks for your reply. Yes the axidma application is a baremetal 
> > > > > application from the xilinx vitis ide example. 
> > > > > Also yes, I have already run hello world as DomU guest, with UART1.
> > > > >
> > > > > I am using the simple mode using polling and not interrupts(they are 
> > > > > disabled in the c code), which is Why I didn't include in the
> > > guest
> > > > > config file. This works well when I compile it normally (as EL3). 
> > > > > When compiled as hypervisor guest, the execution stops before the
> > > > > function xil_dcacheflushrange(). Please find the c code attached
> > > > > in 
> > > > > https://github.com/Xilinx/embeddedsw/blob/master/XilinxProcessorIPLib/drivers/axidma/examples/xaxidma_example_simple_poll.c
> > > > >  
> > > > >
> > > > > I found the SATA example similar to my use case, also using DMA and 
> > > > > SMMU. I just used it as a reference to create my pl.dtb to pass
> > > > through
> > > > > in the guest config.(from pl.dtsi, which I manually created).
> > > > >
> > > > > Thanks and Best Regards,
> > > > > Prem
> > > > >
> > > > > Get Outlook for Android
> > > > >
> >>>>_______________________________________________________________________________________________________________________________________
> _
> > _
> > > _
> > > > _
> > > > > From: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > > > > Sent: Friday 14 May 2021, 23:16
> > > > > To: Prem Harikrishnan
> > > > > Cc: Brian Woods; Stefano Stabellini; Julien Grall; Stefano 
> > > > > Stabellini; xen-users@xxxxxxxxxxxxxxxxxxxx; fnuv@xxxxxxxxxx;
> > > > edgari@xxxxxxxxxx;
> > > > > bwoods288@xxxxxxxxx; Renato Costa Amorim; JOVITAC@xxxxxxxxxx
> > > > > Subject: Re: Help: Information required for AXI DMA passthrough for 
> > > > > PL masters in ZCU102 dev kit
> > > > >
> > > > > I take that the guest kernel "/bin/xaxidma_example_simple_poll_1.bin" 
> > > > > is
> > > > > a baremetal application? Can you run it successfully, just printing
> > > > > "hello world", if you only assign the UART and nothing else?
> > > > >
> > > > > I am asking just to verify that you can run the baremetal VM
> > > > > successfully, aside from the PL device assignment.
> > > > >
> > > > >
> > > > > In regards to the PL device assignment, I'll let Brian answer on the
> > > > > Vivado configuration and SMID calculation. But I noticed that you only
> > > > > assigned IRQ 54 to the DomU which is the one for UART1. You haven't
> > > > > assigned any IRQs for axi_dma_0. Looking at pl_from_petalinux.dtsi, it
> > > > > looks like there are two related IRQs: 121 and 122.
> > > > >
> > > > > Finally, why did you mention SATA passthrough in your reply? I don't
> > > > > follow: I cannot see any SATA controller assignment in your
> > > > > configuration.
> > > > >
> > > > >
> > > > > On Fri, 14 May 2021, Prem Harikrishnan wrote:
> > > > > > Dear Brian, Stefano and Julien,
> > > > > >
> > > > > > Thank you very much for your replies.
> > > > > > Hi Brian and Stefano, Can you please help me with the following 
> > > > > > questions?
> > > > > >
> > > > > > @Brian Woods Please find attached my vivado configuration, when i 
> > > > > > added my AXI DMA now running on simple polling mode(previously
> i
> > > > tried
> > > > > it
> > > > > > with Scatter Gather mode), Run Automation automatically generated, 
> > > > > > 1 AXI Interconnect and 1 smart connect. Since there is only 1
> > > master
> > > > > and
> > > > > > I use the S_AXI _HP0_FPD, I finally get a value of 0xE80 for the 
> > > > > > stream IDs.
> > > > > > https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842066/Xen+and+PL+Masters
> > > > > >  According to this link section 6.5 I have
> > manually
> > > > tied
> > > > > > of AxPROT bits to value of 0x2 to create non secure transactions. 
> > > > > > Can you please confirm that my Vivado configuration is correct?
> I
> > > can
> > > > > > update you soon with an ILA trace on Vivado.
> > > > > >
> > > > > > @Stefano Stabellini Please find attached my pl.dtsi generated from 
> > > > > > petalinux and system.dtsi also generated after manually adding
> > > > > xen.dtsi.
> > > > > > https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/384663561/Building+Xen+Hypervisor+with+Petalinux+2020.1+and+2020.2
> > > > > >  Using
> this
> > > > SATA
> > > > > > passthrough, i made my own pl.dtsi for passthrough used in dma.cfg. 
> > > > > > I could really use your help to identify what is missing in
> my
> > > xen
> > > > > > configuration.
> > > > > >
> > > > > > Thanks and Best Regards,
> > > > > > Prem
> > > > > >
> >>>>>______________________________________________________________________________________________________________________________________
> _
> > _
> > > _
> > > > _
> > > > > _
> > > > > > From: Brian Woods <brian.woods@xxxxxxxxxx>
> > > > > > Sent: Thursday, May 13, 2021 17:13
> > > > > > To: Stefano Stabellini <stefano.stabellini@xxxxxxxxxx>
> > > > > > Cc: Julien Grall <julien@xxxxxxx>; Prem Harikrishnan 
> > > > > > <prem.harikrishnan@xxxxxxxxxxxxxx>; Stefano Stabellini
> > <sstabellini@xxxxxxxxxx>;
> > > > > > xen-users@xxxxxxxxxxxxxxxxxxxx <xen-users@xxxxxxxxxxxxxxxxxxxx>; 
> > > > > > brian.woods@xxxxxxxxxx <brian.woods@xxxxxxxxxx>; fnuv@xxxxxxxxxx
> > > > > > <fnuv@xxxxxxxxxx>; edgari@xxxxxxxxxx <edgari@xxxxxxxxxx>; 
> > > > > > bwoods288@xxxxxxxxx <bwoods288@xxxxxxxxx>
> > > > > > Subject: Re: Help: Information required for AXI DMA passthrough for 
> > > > > > PL masters in ZCU102 dev kit  
> > > > > > Prem,
> > > > > >
> > > > > > Sorry for the late email, I've been under the weather and have been 
> > > > > > away
> > > > > > from my computer.  In the Zynq SOCs this is how the SMMU IDs in PL 
> > > > > > are
> > > > > > calculated:
> > > > > >
> > > > > > 14-10: TBU bits
> > > > > > 9-6: Master ID bits
> > > > > > 5-0: AXI ID bits
> > > > > >
> > > > > > https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf
> > > > > > To look up the TBU value, see UG1085 and look for the "System 
> > > > > > Masters"
> > > > > > table which is table 16-4 on page 409.
> > > > > >
> > > > > > For the master ID, see UG1085 and look for the "Master IDs List" 
> > > > > > table
> > > > > > which is table 16-13 on page 429.
> > > > > >
> > > > > > The AXI ID bits is a bit trickery since it's going to be out in PL.
> > > > > > See: https://www.xilinx.com/support/answers/69447.html
> > > > > > With SmartConnect all the AXI IDs are 0, they aren't passed on.  
> > > > > > There
> > > > > > are work arounds.  Here's a couple of examples (with example 
> > > > > > numbers):
> > > > > >
> > > > > > AXI master -> PS slave port 0
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000001
> > > > > >
> > > > > > AXI master -> SmartConnect -> PS slave port 0
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000000
> > > > > >
> > > > > > AXI master -> SmartConnect -> PS slave port 0
> > > > > >                            -> PS slave port 1
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000000
> > > > > >         TBU bits: 00001, Master ID bits: 0001, AXI ID bits: 0000000
> > > > > >
> > > > > > Two AXI masters -> SmartConnect -> PS slave port 0
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000000 
> > > > > > (both share)
> > > > > >
> > > > > > Two AXI masters -> SmartConnect -> PS slave port 0
> > > > > >                                 -> PS slave port 1
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000000 
> > > > > > (both share)
> > > > > >         TBU bits: 00001, Master ID bits: 0001, AXI ID bits: 0000000 
> > > > > > (both share)
> > > > > >
> > > > > > Two AXI masters -> sideband -> SmartConnect -> sideband -> PS slave 
> > > > > > port 0
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000001 
> > > > > > (master #1)
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000002 
> > > > > > (master #2)
> > > > > >
> > > > > > Two AXI masters -> sideband -> SmartConnect -> sideband -> PS slave 
> > > > > > port 0
> > > > > >                                                         -> PS slave 
> > > > > > port 1
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000001 
> > > > > > (master #1)
> > > > > >         TBU bits: 00001, Master ID bits: 0001, AXI ID bits: 0000001 
> > > > > > (master #1)
> > > > > >         TBU bits: 00001, Master ID bits: 0000, AXI ID bits: 0000002 
> > > > > > (master #2)
> > > > > >         TBU bits: 00001, Master ID bits: 0001, AXI ID bits: 0000002 
> > > > > > (master #2)
> > > > > >
> > > > > > The "sideband -> SmartConenct -> sideband" can also be functionally
> > > > > > replaced with "AXI Interconnect" to my knowledge.  So the AXI ID 
> > > > > > bits are
> > > > > > fairly simple but with some caveats.
> > > > > >
> > > > > > You can also read in the SMMU faults from Xen and use:
> > > > > > https://static.docs.arm.com/ihi0062/dc/IHI0062D_c_system_mmu_architecture_specification.pdf
> > > > > > to decode them. That should tell you the SMMU IDs as well.
> > > > > >
> > > > > > Let me know if this isn't clear or you have any other questions.
> > > > > >
> > > > > > Brian
> > > > > >
> > > > > > On Mon, May 10, 2021 at 06:49:07PM -0700, Stefano Stabellini wrote:
> > > > > > > On Mon, 10 May 2021, Julien Grall wrote:
> > > > > > > > On 26/04/2021 15:41, Prem Harikrishnan wrote:
> > > > > > > > > Hello everyone,
> > > > > > > >
> > > > > > > > Hi Prem,
> > > > > > > >
> > > > > > > > > I am using the latest version of the xilinx mpsoc zcu102 
> > > > > > > > > development board
> > > > > > > > > and i am trying to create a barematal application that uses 
> > > > > > > > > the AXI DMA on
> > > > > > > > > the programmable logic, using the scatter gather example from 
> > > > > > > > > the xilinx
> > > > > > > > > repository. This works perfectly well for the baremetal 
> > > > > > > > > application. The
> > > > > > > > > diagram below represents the information flow. The AXI DMA 
> > > > > > > > > and the AXI data
> > > > > > > > > FIFO are in the Programable logic (FPGA). This AXI DMA is out 
> > > > > > > > > of the box IP
> > > > > > > > > from Vivado and uses a single channel (Does not use AxUSER) 
> > > > > > > > > which means the
> > > > > > > > > AXI ID cannot manually be set.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Now i want to make this as a baremetal hypervisor guest, in 
> > > > > > > > > future this will
> > > > > > > > > be replaced by a custom IP with cache colouring enabled. I 
> > > > > > > > > want to pass
> > > > > > > > > through the AXI DMA in the PL. (Not to be used by Dom0)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I followed this tutorial Baremetal + DomU
> > > > > > > > > <https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842536/XEN+EL1+Baremetal+DomU>
> > > > > > > > > guest changed the base address to 0x4000000,(1GB allocated to 
> > > > > > > > > Dom-0) to
> > > > > > > > > create my EL1 binary. I used Petalinux 2020.2 to generate my 
> > > > > > > > > linux, xen
> > > > > > > > > image and use tftp to boot.
> > > > > > > > >
> > > > > > > > >  From what i understand from the xilinx tutorials 
> > > > > > > > >Xen+and+PL+Masters
> > > > > > > > > <https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18842066/Xen+and+PL+Masters>
> > > > > > > > > section 3 and MPSOC SMMU
> > > > > > > > > <https://xilinx-wiki.atlassian.net/wiki/spaces/A/pages/18841981/Zynq+UltraScale+MPSOC+SMMU>
> > > > > > > > > section 4, stream ids are required by the SMMU to get the 
> > > > > > > > > correct address
> > > > > > > > > translation.
> > > > > > > > >
> > > > > > > > > I am still unable to figure out how to get the iommu property 
> > > > > > > > > for my AXI DMA
> > > > > > > > > and how to set the dtdev config for my hypervisor guest.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I can also see some new updates with the XEN 2020.1 and 
> > > > > > > > > 2020.2 releases
> > > > > > > > > which i can be useful to my application
> > > > > > > > >
> > > > > > > > >   *   1:1 memory mappings for Xen virtual machines, enabling 
> > > > > > > > >device
> > > > > > > > >     assignments without SMMU
> > > > > > > > >   *
> > > > > > > > >       Static assignment of PL blocks, including bus-mastering 
> > > > > > > > >blocks, to
> > > > > > > > >     Xen virtual machines
> > > > > > > > >   *   Support for PL Device Passthrough to DomU in Xen
> > > > > > > >
> > > > > > > > You are usiong a version of Xen that has been customised by 
> > > > > > > > Xilinx. So I would
> > > > > > > > recommend to contact them directly.
> > > > > > > >
> > > > > > > > I have CCed Stefano who might be able to help you.
> > > > > > >
> > > > > > > Hi Julien,
> > > > > > >
> > > > > > > Many thanks for CC'ing me, I missed the email.
> > > > > > >
> > > > > > >
> > > > > > > Hi Prem,
> > > > > > >
> > > > > > > Let me CC a few people that might be able to help.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >
> 
>