Re: [Xen-devel] [RFC PATCH 0/7] Intel Cache Monitoring: Current Status and Future Opportunities

[Meng Xu <xumengpanda@xxxxxxxxx>]

Hi Dario,

2015-04-03 22:14 GMT-04:00 Dario Faggioli <dario.faggioli@xxxxxxxxxx>:
> Hi Everyone,
>
> This RFC series is the outcome of an investigation I've been doing about
> whether we can take better advantage of features like Intel CMT (and of PSR
> features in general). By "take better advantage of" them I mean, for example,
> use the data obtained from monitoring within the scheduler and/or within
> libxl's automatic NUMA placement algorithm, or similar.
>
> I'm putting here in the cover letter a markdown document I wrote to better
> describe my findings and ideas (sorry if it's a bit long! :-D). You can also
> fetch it at the following links:
>
>  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.pdf
>  * http://xenbits.xen.org/people/dariof/CMT-in-scheduling.markdown
>
> See the document itself and the changelog of the various patches for details.
>
> The series includes one Chao's patch on top, as I found it convenient to build
> on top of it. The series itself is available here:
>
>   git://xenbits.xen.org/people/dariof/xen.git  wip/sched/icachemon
>   
> http://xenbits.xen.org/gitweb/?p=people/dariof/xen.git;a=shortlog;h=refs/heads/wip/sched/icachemon
>
> Thanks a lot to everyone that will read and reply! :-)
>
> Regards,
> Dario
> ---
>
> # Intel Cache Monitoring: Present and Future
>
> ## About this document
>
> This document represents the result of in investigation on whether it would be
> possible to more extensively exploit the Platform Shared Resource Monitoring
> (PSR) capabilities of recent Intel x86 server chips. Examples of such features
> are the Cache Monitoring Technology (CMT) and the Memory Bandwidth Monitoring
> (MBM).
>
> More specifically, it focuses on Cache Monitoring Technology, support for 
> which
> has recently been introduced in Xen by Intel, trying to figure out whether it
> can be used for high level load balancing, such as libxl automatic domain
> placement, and/or within Xen vCPU scheduler(s).
>
> Note that, although the document only speaks about CMT, most of the
> considerations apply (or can easily be extended) to MBM as well.
>
> The fact that, currently, support is provided for monitoring L3 cache only,
> somewhat limits the benefits of more extensively exploiting such technology,
> which is exactly the purpose here. Nevertheless, some improvements are 
> possible
> already, and if at some point support for monitoring other cache layers will 
> be
> available, this can be the basic building block for taking advantage of that
> too.

I'm wondering if you really want to know the cache usage at different
levels of cache, you may use the (4) general PMC on each logical core
to monitor that. This could bypass the limitation of the current HW,
but the concern is that it may affect the other mechanisms in Xen,
like perf, which also use the PMC.)

Another thought on the CMT is that it seems that Intel introduces CMT
along with CAT. So I assume they want to use CMT along with CAT so
that it gives some hint on how to allocate LLC to different guests?
For example, if a crazy guest is thrashing the LLC, they can apply CAT
to constraint/calm down this crazy guest.


Best,

Meng

-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel