Re: Xen on Pi4: Xen doesn't work with overlays from Raspberry Pi 5.4 kernel

[Corey Minyard <cminyard@xxxxxxxxxx>]

On Mon, Jun 15, 2020 at 05:14:21PM -0700, Stefano Stabellini wrote:
> On Mon, 15 Jun 2020, Christopher Clark wrote:
> > On Wed, Jun 10, 2020 at 7:21 PM Roman Shaposhnik <roman@xxxxxxxxxx> wrote:
> > >
> > > On Wed, Jun 10, 2020 at 11:54 AM Corey Minyard <cminyard@xxxxxxxxxx> 
> > > wrote:
> > > >
> > > > I had been working on Xen on the Pi4 by throwing kernels I compiled onto
> > > > existing sd cards, and this was working fine.  I finally got to a full
> > > > yocto build of the system, and it didn't boot.
> > > >
> > > > In fact, Xen didn't print anything at all, and nothing happens that
> > > > might suggest it's booting without any console output.
> > 
> > I've reproduced this. Linux 4.19 from the Raspberry Pi kernel branch
> > works fine, whereas I see no console output from the kernel once Xen
> > tries to hand off to dom0 with either a 5.4 or 5.6 kernel.
> > 
> > > > I traced the issue down to the vc4-fkms-v3d dtoverly.  With everything
> > > > else the same, the 4.19 version of that overlay works, and the 5.4
> > > > version does not work.  It also didn't work if I completely removed the
> > > > overlay.  The base device trees are the same between the two kernels.
> > > >
> > > > Looking at the overlay changes between the versions and Xen source, I
> > > > can't trace down anything that would cause an issue.  Has anyone seen
> > > > this issue of have any ideas?
> > 
> > Seen it: yes, but no progress on resolving it to report at this point.
> > 
> > > FWIW: I ran into very similar issues, ditched 5.4 kernel and moved to 
> > > 5.6.x:
> > >     https://github.com/raspberrypi/linux/tree/rpi-5.6.y
> > >
> > > The 5.6.14 seems to be working quite nicely with Xen for me (and Stefano).
> > 
> > Hi Roman - is there a specific commit in that rpi-5.6.y branch that
> > you guys have working ok?
> > It looks like the bcm2711_defconfig file wasn't included in the kernel
> > source tree of that branch at the point the kernel version was bumped
> > up to 5.6.14, so is there somewhere else to look for a matching kernel
> > config?
> 
> I don't know if that is the issue but beware that some device trees
> invert serial0 with serial1. Make sure to use /soc/serial@7e215040. You
> can do that by passing dtuart=/soc/serial@7e215040 to the Xen command
> line.

I already have that set as part of the boot process, but it still
doesn't print anything out once Xen is started.

I tried the 5.6 device tree, and no help there, eithers.  I'm wondering
if everyone is still running with the 4.19 device trees.

The device tree differences between 4.19 and 5.4 are rather large,
unfortunately.

-corey