Re: [Xen-devel] [PATCH RFC LINUX v1] xen: arm: enable migration on ARM.

[Ian Campbell <ian.campbell@xxxxxxxxxx>]

On Wed, 2016-01-06 at 17:55 +0000, Stefano Stabellini wrote:
> Please CC linux-arm for the non-RFC patches
> 
> On Wed, 9 Dec 2015, Ian Campbell wrote:
> > Replace various stub functions with real functionality, including
> > reestablishing the shared info page and the per-vcpu info pages on
> > restore.
> > 
> > Reestablishing the vcpu info page is a little subtle. The
> > VCPUOP_register_vcpu_info hypercall can only be called on either the
> > current VCPU or on an offline different VCPU. Since migration occurs
> > with all VCPUS online they are all therefore online at the point of
> > resume.
> > 
> > Therefore we must perform a cross VCPU call to each non-boot VCPU,
> > which cannot be done in the xen_arch_post_suspend() callback since
> > that is run from stop_machine() with interrupts disabled.
> > 
> > Furthermore VCPUOP_register_vcpu_info can only be called once per-VCPU
> > in a given domain, so it must not be called after a cancelled suspend
> > (which resumes in the same domain).
> > 
> > Therefore xen_arch_resume() gains a suspend_cancelled parameter and we
> > resume the secondary VCPUs there only if needed.
> 
> It is a bit complex but it seems better than what we do on x86, which
> is:

We cannot do this on ARM in any case, since we can't do VCPUOP_{up,down}
and we don't want to expose this. We might be able to do it via PCSI but
TBH I dislike this anyway since it defeats the purpose of migrating with
all VCPUs online, we may as well just bring them down before and bring them
back up afterwards (as we used to before the tools supported multivcpu
migration, i.e. the $dom/control/platform-feature-multiprocessor-suspend
node in xs).
> 
>       for_each_possible_cpu(cpu) {
>               bool other_cpu = (cpu != smp_processor_id());
>               bool is_up = HYPERVISOR_vcpu_op(VCPUOP_is_up, cpu, NULL);
> 
>               if (other_cpu && is_up &&
>               ÂÂÂÂHYPERVISOR_vcpu_op(VCPUOP_down, cpu, NULL))
>                       BUG();
> 
>               xen_setup_runstate_info(cpu);
> 
>               if (have_vcpu_info_placement)
>                       xen_vcpu_setup(cpu);
> 
>               if (other_cpu && is_up &&
>               ÂÂÂÂHYPERVISOR_vcpu_op(VCPUOP_up, cpu, NULL))
>                       BUG();
>       }
> 
>Â
> > +void xen_arch_post_suspend(int suspend_cancelled)
> > +{
> > +   xen_register_shared_info();
> > +   if (!suspend_cancelled)
> > +           xen_vcpu_restore();
> 
> could we wait and call xen_vcpu_restore for cpu0 from
> xen_vcpu_notify_resume?

I've been through almost every permutation of where this stuff can be done.
IIRC we need to take some events on this vcpu before then, else we get
wedged before the notify.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel