Re: Xen on Pi4: Xen doesn't work with overlays from Raspberry Pi 5.4 kernel

[Roman Shaposhnik <roman@xxxxxxxxxx>]

On Mon, Jun 15, 2020 at 6:02 PM Corey Minyard <cminyard@xxxxxxxxxx> wrote:
>
> On Mon, Jun 15, 2020 at 05:14:21PM -0700, Stefano Stabellini wrote:
> > On Mon, 15 Jun 2020, Christopher Clark wrote:
> > > On Wed, Jun 10, 2020 at 7:21 PM Roman Shaposhnik <roman@xxxxxxxxxx> wrote:
> > > >
> > > > On Wed, Jun 10, 2020 at 11:54 AM Corey Minyard <cminyard@xxxxxxxxxx> 
> > > > wrote:
> > > > >
> > > > > I had been working on Xen on the Pi4 by throwing kernels I compiled 
> > > > > onto
> > > > > existing sd cards, and this was working fine.  I finally got to a full
> > > > > yocto build of the system, and it didn't boot.
> > > > >
> > > > > In fact, Xen didn't print anything at all, and nothing happens that
> > > > > might suggest it's booting without any console output.
> > >
> > > I've reproduced this. Linux 4.19 from the Raspberry Pi kernel branch
> > > works fine, whereas I see no console output from the kernel once Xen
> > > tries to hand off to dom0 with either a 5.4 or 5.6 kernel.
> > >
> > > > > I traced the issue down to the vc4-fkms-v3d dtoverly.  With everything
> > > > > else the same, the 4.19 version of that overlay works, and the 5.4
> > > > > version does not work.  It also didn't work if I completely removed 
> > > > > the
> > > > > overlay.  The base device trees are the same between the two kernels.
> > > > >
> > > > > Looking at the overlay changes between the versions and Xen source, I
> > > > > can't trace down anything that would cause an issue.  Has anyone seen
> > > > > this issue of have any ideas?
> > >
> > > Seen it: yes, but no progress on resolving it to report at this point.
> > >
> > > > FWIW: I ran into very similar issues, ditched 5.4 kernel and moved to 
> > > > 5.6.x:
> > > >     https://github.com/raspberrypi/linux/tree/rpi-5.6.y
> > > >
> > > > The 5.6.14 seems to be working quite nicely with Xen for me (and 
> > > > Stefano).
> > >
> > > Hi Roman - is there a specific commit in that rpi-5.6.y branch that
> > > you guys have working ok?
> > > It looks like the bcm2711_defconfig file wasn't included in the kernel
> > > source tree of that branch at the point the kernel version was bumped
> > > up to 5.6.14, so is there somewhere else to look for a matching kernel
> > > config?
> >
> > I don't know if that is the issue but beware that some device trees
> > invert serial0 with serial1. Make sure to use /soc/serial@7e215040. You
> > can do that by passing dtuart=/soc/serial@7e215040 to the Xen command
> > line.
>
> I already have that set as part of the boot process, but it still
> doesn't print anything out once Xen is started.
>
> I tried the 5.6 device tree, and no help there, eithers.  I'm wondering
> if everyone is still running with the 4.19 device trees.

As I pointed out in the EVE link above -- we're very happily running
with 5.6 device trees. They are, of course, taken from RPI kernel
tree.

Thanks,
Roman.