[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 4/8] xen/common: Split tlb-clock.h out of mm.h


  • To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Thu, 13 Mar 2025 13:59:13 +0100
  • Autocrypt: addr=jbeulich@xxxxxxxx; keydata= xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A nAuWpQkjM1ASeQwSHEeAWPgskBQL
  • Cc: Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Julien Grall <julien@xxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>, Bertrand Marquis <bertrand.marquis@xxxxxxx>, Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>, Shawn Anastasio <sanastasio@xxxxxxxxxxxxxxxxxxxxx>, Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>
  • Delivery-date: Thu, 13 Mar 2025 12:59:24 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 12.03.2025 18:45, Andrew Cooper wrote:
> xen/mm.h includes asm/tlbflush.h almost at the end, which creates a horrible
> tangle.  This is in order to provide two common files with an abstraction over
> the x86-specific TLB clock logic.
> 
> First, introduce CONFIG_HAS_TLB_CLOCK, selected by x86 only.  Next, introduce
> xen/tlb-clock.h, providing empty stubs, and include this into memory.c and
> page_alloc.c
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> ---
> CC: Anthony PERARD <anthony.perard@xxxxxxxxxx>
> CC: Michal Orzel <michal.orzel@xxxxxxx>
> CC: Jan Beulich <jbeulich@xxxxxxxx>
> CC: Julien Grall <julien@xxxxxxx>
> CC: Roger Pau Monné <roger.pau@xxxxxxxxxx>
> CC: Stefano Stabellini <sstabellini@xxxxxxxxxx>
> CC: Volodymyr Babchuk <Volodymyr_Babchuk@xxxxxxxx>
> CC: Bertrand Marquis <bertrand.marquis@xxxxxxx>
> CC: Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>
> CC: Shawn Anastasio <sanastasio@xxxxxxxxxxxxxxxxxxxxx>
> 
> There is still a mess here with the common vs x86 split, but it's better
> contained than before.
> ---
>  xen/arch/x86/Kconfig        |  1 +
>  xen/common/Kconfig          |  3 +++
>  xen/common/memory.c         |  1 +
>  xen/common/page_alloc.c     |  1 +
>  xen/include/xen/mm.h        | 27 --------------------
>  xen/include/xen/tlb-clock.h | 49 +++++++++++++++++++++++++++++++++++++
>  6 files changed, 55 insertions(+), 27 deletions(-)
>  create mode 100644 xen/include/xen/tlb-clock.h
> 
> diff --git a/xen/arch/x86/Kconfig b/xen/arch/x86/Kconfig
> index 6e41bc0fb435..e9a166ee3dd0 100644
> --- a/xen/arch/x86/Kconfig
> +++ b/xen/arch/x86/Kconfig
> @@ -28,6 +28,7 @@ config X86
>       select HAS_PCI_MSI
>       select HAS_PIRQ
>       select HAS_SCHED_GRANULARITY
> +     select HAS_TLB_CLOCK
>       select HAS_UBSAN
>       select HAS_VMAP
>       select HAS_VPCI if HVM
> diff --git a/xen/common/Kconfig b/xen/common/Kconfig
> index 6166327f4d14..dcf7d9d00d0a 100644
> --- a/xen/common/Kconfig
> +++ b/xen/common/Kconfig
> @@ -83,6 +83,9 @@ config HAS_PMAP
>  config HAS_SCHED_GRANULARITY
>       bool
>  
> +config HAS_TLB_CLOCK
> +     bool
> +
>  config HAS_UBSAN
>       bool
>  
> diff --git a/xen/common/memory.c b/xen/common/memory.c
> index 61a94b23abae..9138fd096696 100644
> --- a/xen/common/memory.c
> +++ b/xen/common/memory.c
> @@ -24,6 +24,7 @@
>  #include <xen/perfc.h>
>  #include <xen/sched.h>
>  #include <xen/sections.h>
> +#include <xen/tlb-clock.h>
>  #include <xen/trace.h>
>  
>  #include <asm/current.h>
> diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c
> index bc029ea797a2..b90c3d7988b4 100644
> --- a/xen/common/page_alloc.c
> +++ b/xen/common/page_alloc.c
> @@ -137,6 +137,7 @@
>  #include <xen/sections.h>
>  #include <xen/softirq.h>
>  #include <xen/spinlock.h>
> +#include <xen/tlb-clock.h>
>  #include <xen/vm_event.h>
>  #include <xen/xvmalloc.h>
>  
> diff --git a/xen/include/xen/mm.h b/xen/include/xen/mm.h
> index 45000cc3f64b..fff36ff903d6 100644
> --- a/xen/include/xen/mm.h
> +++ b/xen/include/xen/mm.h
> @@ -588,33 +588,6 @@ unsigned long get_upper_mfn_bound(void);
>  
>  #include <asm/flushtlb.h>
>  
> -static inline void accumulate_tlbflush(bool *need_tlbflush,
> -                                       const struct page_info *page,
> -                                       uint32_t *tlbflush_timestamp)
> -{
> -    if ( page->u.free.need_tlbflush &&
> -         page->tlbflush_timestamp <= tlbflush_current_time() &&
> -         (!*need_tlbflush ||
> -          page->tlbflush_timestamp > *tlbflush_timestamp) )
> -    {
> -        *need_tlbflush = true;
> -        *tlbflush_timestamp = page->tlbflush_timestamp;
> -    }
> -}
> -
> -static inline void filtered_flush_tlb_mask(uint32_t tlbflush_timestamp)
> -{
> -    cpumask_t mask;
> -
> -    cpumask_copy(&mask, &cpu_online_map);
> -    tlbflush_filter(&mask, tlbflush_timestamp);
> -    if ( !cpumask_empty(&mask) )
> -    {
> -        perfc_incr(need_flush_tlb_flush);
> -        arch_flush_tlb_mask(&mask);
> -    }
> -}
> -
>  enum XENSHARE_flags {
>      SHARE_rw,
>      SHARE_ro,
> diff --git a/xen/include/xen/tlb-clock.h b/xen/include/xen/tlb-clock.h
> new file mode 100644
> index 000000000000..796c0be7fbef
> --- /dev/null
> +++ b/xen/include/xen/tlb-clock.h
> @@ -0,0 +1,49 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef XEN_TLB_CLOCK_H
> +#define XEN_TLB_CLOCK_H
> +
> +#include <xen/types.h>
> +
> +#ifdef CONFIG_HAS_TLB_CLOCK
> +
> +#include <xen/mm.h>
> +
> +#include <asm/flushtlb.h>
> +
> +static inline void accumulate_tlbflush(
> +    bool *need_tlbflush, const struct page_info *page,
> +    uint32_t *tlbflush_timestamp)
> +{
> +    if ( page->u.free.need_tlbflush &&
> +         page->tlbflush_timestamp <= tlbflush_current_time() &&
> +         (!*need_tlbflush ||
> +          page->tlbflush_timestamp > *tlbflush_timestamp) )
> +    {
> +        *need_tlbflush = true;
> +        *tlbflush_timestamp = page->tlbflush_timestamp;
> +    }
> +}
> +
> +static inline void filtered_flush_tlb_mask(uint32_t tlbflush_timestamp)
> +{
> +    cpumask_t mask;
> +
> +    cpumask_copy(&mask, &cpu_online_map);
> +    tlbflush_filter(&mask, tlbflush_timestamp);
> +    if ( !cpumask_empty(&mask) )
> +    {
> +        perfc_incr(need_flush_tlb_flush);

Would this perf counter then perhaps also want to become dependent upon
HAS_TLB_CLOCK=y (or become an arch-specific one)? It's used elsewhere in x86,
but not for any of the other arch-es.

However, see below.

> +        arch_flush_tlb_mask(&mask);
> +    }
> +}
> +
> +#else /* !CONFIG_HAS_TLB_CLOCK */
> +
> +struct page_info;
> +static inline void accumulate_tlbflush(
> +    bool *need_tlbflush, const struct page_info *page,
> +    uint32_t *tlbflush_timestamp) {}
> +static inline void filtered_flush_tlb_mask(uint32_t tlbflush_timestamp) {}

Is doing nothing here correct? mark_page_free() can set a page's
->u.free.need_tlbflush. And with that flag set the full

static inline void accumulate_tlbflush(
    bool *need_tlbflush, const struct page_info *page,
    uint32_t *tlbflush_timestamp)
{
    if ( page->u.free.need_tlbflush &&
         page->tlbflush_timestamp <= tlbflush_current_time() &&
         (!*need_tlbflush ||
          page->tlbflush_timestamp > *tlbflush_timestamp) )
    {
        *need_tlbflush = true;
        *tlbflush_timestamp = page->tlbflush_timestamp;
    }
}

reduces to (considering that tlbflush_current_time() resolves to constant 0,
which also implies every page's ->tlbflush_timestamp is only ever 0)

static inline void accumulate_tlbflush(
    bool *need_tlbflush, const struct page_info *page,
    uint32_t *tlbflush_timestamp)
{
    if ( !*need_tlbflush )
        *need_tlbflush = true;
}

which means a not-stubbed-out filtered_flush_tlb_mask(), with tlbflush_filter()
doing nothing, would actually invoke arch_flush_tlb_mask() (with all online CPUs
set in the mask) when called. And arch_flush_tlb_mask() isn't a no-op on Arm.

I therefore think that while moving stuff into a separate header makes sense,
HAS_TLB_CLOCK isn't overly useful to introduce.

Jan



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.