Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

[Corey Minyard <minyard@xxxxxxx>]

On Wed, Jun 03, 2020 at 08:37:09AM -0700, Stefano Stabellini wrote:
> On Wed, 3 Jun 2020, Corey Minyard wrote:
> > On Tue, Jun 02, 2020 at 12:24:05PM -0700, Stefano Stabellini wrote:
> > > On Tue, 2 Jun 2020, Corey Minyard wrote:
> > > > Snip
> > > > 
> > > > > > > > > whether
> > > > > > > > > this was already done:
> > > > > > > > >      1) Does the kernel boot on baremetal (i.e without Xen)? 
> > > > > > > > > This should
> > > > > > > > > help
> > > > > > > > > to confirm whether the bug is Xen is related.
> > > > > > > > 
> > > > > > > > Yes it boots
> > > > > > > > 
> > > > > > > > >      2) Swiotlb should not be necessary for basic dom0 boot 
> > > > > > > > > on Arm. Did
> > > > > > > > > you try
> > > > > > > > > to disable it? This should help to confirm whether swiotlb is 
> > > > > > > > > the
> > > > > > > > > problem or
> > > > > > > > > not.
> > > > > > > > 
> > > > > > > > It boots disabling swiotlb-xen
> > > > > > > 
> > > > > > > Thank you for the answer! swiotlb-xen should basically be a NOP 
> > > > > > > for dom0. So
> > > > > > > this suggests swiotlb is doing some transformation on the DMA 
> > > > > > > address.
> > > > > > > 
> > > > > > > I have an idea what may have gone wrong. AFAICT, xen-swiotlb 
> > > > > > > seems to assume
> > > > > > > the DMA address space and CPU address space is the same. Is RPI 
> > > > > > > using the
> > > > > > > same address space?
> > > > > > 
> > > > > > Another question, is the DMA request bounced? If so, are we sure 
> > > > > > the bounce
> > > > > > buffer is in the first GB?
> > > > > 
> > > > > Yes, it is. This is actually where we spent the last few days, and I
> > > > > found another little related bug in the initialization of the
> > > > > swiotlb-xen but now I am sure the memory is under 1GB 
> > > > > (0x34000000-0x38000000)
> > > > 
> > > > Was anything ever resolved on this issue?  It just kind of ended for me,
> > > > and I looked in the main kernel and didn't find anything that looked
> > > > related.
> > > 
> > > Yes, we have a patch series on the list for the Linux kernel to fix this
> > > issue but it hasn't been merged yet:
> > > 
> > > https://marc.info/?l=linux-kernel&m=159001831406263&w=2
> > 
> > Just FYI, I pulled the changes on top of
> >   https://github.com/raspberrypi/linux.git rpi-5.4.y
> > Along with change
> >   56e35f9c5b87ec dma-mapping: drop the dev argument to arch_sync_dma_for_*
> > before the series so it applies on 5.4, and I was able to boot and
> > create a domU.  So:
> > 
> > Tested-by: Corey Minyard <cminyard@xxxxxxxxxx>
> > 
> > At least on 5.4.  If you think it would be valuable, I can test on
> > rpi-5.7.y.
> 
> I'd feel better adding your Tested-by to my next upstream submission of
> the series if you could also test on 5.7. Thank you in advance!

Well, rpi-5.7.y fails to bootup completely without Xen, and doesn't even
display any console output on top of Xen :-(.  So there are issues, but
probably not with Xen.

I did try rpi-5.6.y and it works.

-corey

> 
> 
> > I'll be integrating this in with our Pi Xen yocto layer at
> > https://github.com/MontaVista-OpenSourceTechnology/meta-raspberrypi-xen
> 
> That's great!