Re: [Xen-devel] How to boot domU and dom0 from a device tree

[Stefano Stabellini <sstabellini@xxxxxxxxxx>]

On Sat, 15 Jun 2019, Julien Grall wrote:
> Hi Stefano,
> 
> On 6/14/19 9:53 PM, Stefano Stabellini wrote:
> > On Wed, 12 Jun 2019, Julien Grall wrote:
> > > (Moving from xen-users to xen-devel).
> > > 
> > > On 11/06/2019 23:18, Stefano Stabellini wrote:
> > > > I managed to reproduced the issue, and I know how to get past it.  Try
> > > > using the raw kernel Image (arch/arm64/boot/Image) instead of Image.gz
> > > > for dom0 and domU. That fixed it for me.
> > > > 
> > > > Julien, I didn't manage to figure out what the issue is exactly, but it
> > > > looks like Image.gz loading is broken at the moment.
> > > 
> > > Do you mean Image.gz is broken from DomU? Because per the log provided by
> > > Denis, this is working perfectly for Dom0 as we don't create domain in
> > > parallel.
> > > 
> > > By reading the code I can already spot the reason of the first issue
> > > reported
> > > by Denis. For reminder, this is when Dom0 and DomU are using the same
> > > module
> > > address for the gzip Image.
> > > 
> > > This is because when probing the kernel for Dom0, the module will get
> > > uncompressed and the module start/end will be updated to point to the
> > > uncompress version. Because of that, the probe for DomU kernel will not be
> > > able to find the module (the start addressed changed).
> > > 
> > > In this case, I think we only want to uncompress the module one time to
> > > avoid
> > > wasting memory. The solution I have in mind requires some rework in Xen, I
> > > would actually start by probing the information for all the domains, then
> > > uncompress the kernels modules, and then finish to build the domain.
> > > 
> > > For the out of memory problem discussed in this e-mail, I think the
> > > problem is
> > > not because of lack of memory in DomU. The problem is related to the
> > > inflate/gunzip the code. The code is using an heap (see perform_gunzip)
> > > where
> > > it allocates memory from.
> > > 
> > > I am assuming the kernels for Dom0 and DomU are exactly the same but they
> > > are
> > > coming from different address. Am I correct? If so, I am a bit unsure this
> > > worked the first time and not the second time. This probably want some
> > > debugging to understand the problem. Denis, Stefano, can one of you look
> > > at
> > > it?
> > 
> > I couldn't find exactly the root cause yet, but I can reproduce the
> > issue even with Dom0 only (no domUs, no dom0less):
> 
> Looking at Denis's report, the error does not seem to be the same:
> 
> (XEN) ****************************************
> (XEN) Panic on CPU 0:
> (XEN) Out of memory
> (XEN) ****************************************
> 
> 
> But I think they may be related (see below).
> 
> 
> > ee.
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) DEBUG kernel_probe 445
> > (XEN) Loading d0 kernel from boot module @ 0000000047000000
> > (XEN) Loading ramdisk from boot module @ 0000000042000000
> > (XEN) DEBUG kernel_decompress 268
> > (XEN) DEBUG kernel_decompress 272
> > (XEN) DEBUG kernel_decompress 279
> > (XEN) DEBUG kernel_decompress 284
> > (XEN) DEBUG kernel_decompress 291 kernel_order_out=52 output_size=0
> > (XEN)
> > (XEN) ****************************************
> > (XEN) Panic on CPU 0:
> > (XEN) Could not set up DOM0 guest OS
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Reboot in five seconds...
> > 
> > 
> > The issue seems to be that output_size, returned by output_length(input,
> > size) is 0. Then, kernel_order_out is set to 52 which is too large. As a
> > consequence kernel_decompress returns with -ENOMEM.
> 
> I have just tried to use compressed kernel and can't reproduce your error.
> However, I think the two problems ("out of memory" and your one) are because
> the module size does not exactly match the size of the compressed image.
> 
> The uncompressed size is part of the footer (the last 4-bytes). As we only
> have the module size in hand, we assume it is equal to the compressed size. If
> not, then we will return whatever is in the last 4-bytes of the module.
> 
> This means the module size should exactly match the compressed image size.
> AFAICT, gzip format doesn't provide a field for the compressed size, so we
> can't do better in Xen.
> 
> In other word, the Device-Tree multiboot nodes should be created with the
> exact size of the compressed image.

Yes, you are right! That was the cause of the issue I was seeing.
Definitely something to watch out for!


> Regardless that, I still think we have some issues when using the same
> compressed kernel for Dom0 and DomU (see in my previous e-mail).

You wrote in the previous email:

> By reading the code I can already spot the reason of the first issue reported 
> by Denis. For reminder, this is when Dom0 and DomU are using
> the same module address for the gzip Image.

By "module address", do you mean they use the same loading address in
u-boot? Because Denis was loading the Image.gz kernel twice at different
addresses for dom0 and domU: at 0x47000000 and at 0x43000000.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel