Re: Trying add smt=off disabled cores to cpupool crash xen

[Juergen Gross <jgross@xxxxxxxx>]

On 04.12.23 11:13, Jan Beulich wrote:
> On 04.12.2023 11:02, Juergen Gross wrote:
> > On 04.12.23 10:15, Jan Beulich wrote:
> > > On 01.12.2023 21:12, Andrew Cooper wrote:
> > > > On 01/12/2023 7:59 pm, René Winther Højgaard wrote:
> > > > > If I set smt=off and try to configure cpupools with credit(1) as if
> > > > > all cores are available, I get the following crash.
> > > > >
> > > > > The crash happens when I try to use xl cpupool-add-cpu on the disabled
> > > > > HT sibling cores.
> > > > >
> > > > > Hyper-threading is enabled in the firmware, and only disabled with
> > > > > smt=off.
> > > > CC'ing some maintainers.
> > > >
> > > > I expect this will also explode when a CPU is runtime offlined with
> > > > `xen-hptool cpu-offline` and then added to a cpupool.
> > > >
> > > > Interestingly, the crash is mov (%rdx,%rax,1),%r13, and I think that's
> > > > the percpu posion value in %rdx.
> > > >
> > > > I expect cpupools want to reject parked/offline CPUs.
> > > While the only explicit check there is
> > >
> > >           if ( cpu >= nr_cpu_ids )
> > >               goto addcpu_out;
> > >
> > > I would have expected this
> > >
> > >           if ( !cpumask_subset(cpus, &cpupool_free_cpus) ||
> > >                cpumask_intersects(cpus, &cpupool_locked_cpus) )
> > >               goto addcpu_out;
> > >
> > > to deal with the situation, as parked/offline CPUs shouldn't be "free".
> > > Jürgen?
> > The problem is the call of sched_get_opt_cpumask() to need the percpu area
> > of the cpu in question.
> That was my first thought, too, but then I saw cpupool_assign_cpu_locked() on
> the call trace, which is called only afterwards. Plus sched_get_opt_cpumask()
> needs the per-CPU area only when granularity was switched from its default of
> SCHED_GRAN_cpu afaics.
Oh right you are.

My patch is needed for larger granularities, though.

I've tried to hit the same problem as René, but everything works as intended (no
crash, but adding an offline cpu is being rejected).

René, could you please tell us what exactly you've been doing? This would be:

- Xen command line parameters
- Output of "xl info"
- Output of "xl cpupool-list" before starting to manipulate cpupools
- Output of "xl cpupool-list -c" before starting to manipulate cpupools
- Cpupool config file used to create new cpupool
- xl commands you've used to setup the cpupool and adding the cpu(s) to it

Thanks,


Juergen

Attachment: OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature