Re: [Xen-devel] [PATCH v3 3/6] xen: RCU/x86/ARM: discount CPUs that were idle when grace period started.

[George Dunlap <george.dunlap@xxxxxxxxxx>]

On 08/18/2017 07:04 PM, Dario Faggioli wrote:
> Xen is a tickless (micro-)kernel, i.e., when a CPU becomes
> idle there is no timer tick that will periodically wake the
> CPU up.
> OTOH, when we imported RCU from Linux, Linux was (on x86) a
> ticking kernel, i.e., there was a periodic timer tick always
> running, even on idle CPUs. This was bad for power consumption,
> but, for instance, made it easy to monitor the quiescent states
> of all the CPUs, and hence tell when RCU grace periods ended.
> 
> In Xen, that is impossible, and that's particularly problematic
> when the system is very lightly loaded, as some CPUs may never
> have the chance to tell the RCU core logic about their quiescence,
> and grace periods could extend indefinitely!
> 
> This has led, on x86, to long (and unpredictable) delays between
> RCU callbacks queueing and their actual invokation. On ARM, we've
> even seen infinite grace periods (e.g., complate_domain_destroy()
> never being actually invoked!). See here:
> 
>  https://lists.xenproject.org/archives/html/xen-devel/2017-01/msg02454.html
> 
> The first step for fixing this situation is for RCU to record,
> at the beginning of a grace period, which CPUs are already idle.
> In fact, being idle, they can't be in the middle of any read-side
> critical section, and we don't have to wait for their quiescence.
> 
> This is tracked in a cpumask, in a similar way to how it was also
> done in Linux (on s390, which was tickless already). It is also
> basically the same approach used for making Linux x86 tickless,
> in 2.6.21 on (see commit 79bf2bb3 "tick-management: dyntick /
> highres functionality").
> 
> For correctness, wee also add barriers. One is also present in
> Linux, (see commit c3f59023, "Fix RCU race in access of nohz_cpu_mask",
> although, we change the code comment to something that makes better
> sense for us). The other (which is its pair), is put in the newly
> introduced function rcu_idle_enter(), right after updating the
> cpumask. They prevent races between CPUs going idle during the
> beginning of a grace period.
> 
> Signed-off-by: Dario Faggioli <dario.faggioli@xxxxxxxxxx>

Reviewed-by: George Dunlap <george.dunlap@xxxxxxxxxx>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel