Re: [linux-linus test] 183794: regressions - FAIL

[Julien Grall <julien@xxxxxxx>]

Hi Juergen,

On 23/11/2023 05:57, Juergen Gross wrote:
> On 23.11.23 00:07, Stefano Stabellini wrote:
> > On Wed, 22 Nov 2023, Juergen Gross wrote:
> > > On 22.11.23 04:07, Stefano Stabellini wrote:
> > > > On Mon, 20 Nov 2023, Stefano Stabellini wrote:
> > > > > On Mon, 20 Nov 2023, Juergen Gross wrote:
> > > > > > On 20.11.23 03:21, osstest service owner wrote:
> > > > > > > flight 183794 linux-linus real [real]
> > > > > > > http://logs.test-lab.xenproject.org/osstest/logs/183794/
> > > > > > >
> > > > > > > Regressions :-(
> > > > > > >
> > > > > > > Tests which did not succeed and are blocking,
> > > > > > > including tests which could not be run:
> > > > > > >     test-arm64-arm64-examine      8 reboot                   fail 
> > > > > > > REGR.
> > > > > > > vs.
> > > > > > > 183766
> > > > > > I'm seeing the following in the serial log:
> > > > > >
> > > > > > Nov 20 00:25:41.586712 [    0.567318] kernel BUG at
> > > > > > arch/arm64/xen/../../arm/xen/enlighten.c:164!
> > > > > > Nov 20 00:25:41.598711 [    0.574002] Internal error: Oops - BUG:
> > > > > > 00000000f2000800 [#1] PREEMPT SMP
> > > > > >
> > > > > > The related source code lines in the kernel are:
> > > > > >
> > > > > > ········err = HYPERVISOR_vcpu_op(VCPUOP_register_vcpu_info,
> > > > > > xen_vcpu_nr(cpu),
> > > > > > ································ &info);
> > > > > > ········BUG_ON(err);
> > > > > >
> > > > > > I suspect commit 20f3b8eafe0ba to be the culprit.
> > > > > >
> > > > > > Stefano, could you please have a look?
> > > > The good news and bad news is that I cannot repro this neither with nor
> > > > without CONFIG_UNMAP_KERNEL_AT_EL0. I looked at commit 20f3b8eafe0ba 
> > > > but
> > > > I cannot see anything wrong with it. Looking at the register dump, 
> > > > from:
> > > > x0 : fffffffffffffffa
> > > >
> > > > I am guessing the error was -ENXIO which is returned from 
> > > > map_guest_area
> > > > in Xen.
> > > >
> > > > Could it be that the struct is crossing a page boundary? Or that it is
> > > > not 64-bit aligned? Do we need to do something like the following?
> > > >
> > > > diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> > > > index 9afdc4c4a5dc..5326070c5dc0 100644
> > > > --- a/arch/arm/xen/enlighten.c
> > > > +++ b/arch/arm/xen/enlighten.c
> > > > @@ -484,7 +485,7 @@ static int __init xen_guest_init(void)
> > > >         * for secondary CPUs as they are brought up.
> > > >         * For uniformity we use VCPUOP_register_vcpu_info even on cpu0.
> > > >         */
> > > > -    xen_vcpu_info = alloc_percpu(struct vcpu_info);
> > > > +    xen_vcpu_info = __alloc_percpu(struct vcpu_info, PAGE_SIZE);
> > > >        if (xen_vcpu_info == NULL)
> > > >            return -ENOMEM;
> > > May I suggest to use a smaller alignment? What about:
> > >
> > > 1 << fls(sizeof(struct vcpu_info) - 1)
> > See below
> >
> > ---
> > [PATCH] arm/xen: fix xen_vcpu_info allocation alignment
Stefano, are you going to submit the patch formally?

> > xen_vcpu_info is a percpu area than needs to be mapped by Xen.
> > Currently, it could cross a page boundary resulting in Xen being unable
> > to map it:
> >
> > [    0.567318] kernel BUG at 
> > arch/arm64/xen/../../arm/xen/enlighten.c:164!
> > [    0.574002] Internal error: Oops - BUG: 00000000f2000800 [#1] 
> > PREEMPT SMP
> > Fix the issue by using __alloc_percpu and requesting alignment for the
> > memory allocation.
> >
> > Signed-off-by: Stefano Stabellini <stefano.stabellini@xxxxxxx>
I am guessing we want to backport it. So should this contain a tag to 
indicate the intention?
> > diff --git a/arch/arm/xen/enlighten.c b/arch/arm/xen/enlighten.c
> > index 9afdc4c4a5dc..09eb74a07dfc 100644
> > --- a/arch/arm/xen/enlighten.c
> > +++ b/arch/arm/xen/enlighten.c
> > @@ -484,7 +484,8 @@ static int __init xen_guest_init(void)
> >        * for secondary CPUs as they are brought up.
> >        * For uniformity we use VCPUOP_register_vcpu_info even on cpu0.
> >        */
> > -    xen_vcpu_info = alloc_percpu(struct vcpu_info);
> > +    xen_vcpu_info = __alloc_percpu(sizeof(struct vcpu_info),
> > +                                   1 << fls(sizeof(struct vcpu_info) 
> > - 1));
> Nit: one tab less, please (can be fixed while committing).
>
> >       if (xen_vcpu_info == NULL)
> >           return -ENOMEM;
> Reviewed-by: Juergen Gross <jgross@xxxxxxxx>
Juergen, looking at the x86 code, you seem to use DEFINE_PER_CPU(). So 
what guarantees that this is not going to cross a page?
Cheers,

--
Julien Grall