Re: [Xen-devel] [RFC 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document.

[Meng Xu <mengxu@xxxxxxxxxxxxx>]

Hi Yi,

I have some quick comment about this document. Some minor points are
not very clear, IMHO.

> +
> +  2. `psr-mba-set [OPTIONS] domain-id throttling`:
> +
> +     Set memory bandwidth throttling for domain.
> +
> +     Options:
> +     '-s': Specify the socket to process, otherwise all sockets are 
> processed.
> +
> +     Throttling value set in register implies memory bandwidth blocked, i.e.
> +     higher throttling value results in lower bandwidth. The max throttling
> +     value can be got through CPUID.
> +
> +     The response of the throttling value could be linear mode or non-linear
> +     mode.
> +
> +     Linear mode: the input precision is defined as 100-(MBA_MAX). For 
> instance,
> +     if the MBA_MAX value is 90, the input precision is 10%. Values not an 
> even
> +     multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10%
> +     delay applied) by HW automatically.

So MBA has a minimum allocation unit. What is the minimum bandwidth
allocation unit?
From the above example, I had the impression that the allocation unit is 10%.
As mentioned in the document later, the throttle value is set in the
COS register's  Thrtl bit fields as shown in [Code_CBM, Data_CBM,
Thrtl]. I had the impression that the maximum number of bandwidth
units we can allocate is 2^number_of_bits_in_Thrtl.
Only one of my impression could be true, right? ;-)

In addition, since hardware will round down the partial bandwidth
value, why shouldn't we just allow system operators to configure the
"valid" bandwidth supported by the hardware.
For example, if the hardware only supports the bandwidth  throttle
value in 10% units, then we should not allow users to input the
bandwidth throttle value as 12% or 13%. Otherwise, as a system
operator, I would be confused at why I increased the bandwidth
throttle value from 11% to 19%, I still see the same bandwidth
guarantee.

> +
> +     Non-linear mode: input delay values are powers-of-two from zero to the
> +     MBA_MAX value from CPUID. In this case any values not a power of two 
> will
> +     be rounded down the next nearest power of two by HW automatically.\

First question: Why is it the delay value instead of bandwidth value
in the non-linear mode? Does MBA really control memory access latency?

Second question: Does the hardware provide any guaranteed bandwidth in
the non-linear mode?
I saw the document patch in Linux at
http://www.mail-archive.com/linux-kernel@xxxxxxxxxxxxxxx/msg1307176.html:
[Qutoe]
In nonlinear scale currently SDM specifies
+throttle values in 2^n values. However the h/w does not guarantee a
+specific curve for the amount of memory b/w that is actually throttled.
+But for any thrtl_by value x > y, its guaranteed that x would throttle
+more b/w than y.  The info directory specifies the max thrtl_by value
+and thrtl_by granularity.
[/Qutoe]

It seems that the non-linear mode simply provide some throttling
relations but don't guarantee the actual throttle value.
Maybe it will be good to clearly state the capability and limitations
of the hardware.

> +  System administrator can change PSR allocation policy at runtime by
> +  tool stack. Since MBA shares COS with CAT/CDP, a COS corresponds to a
> +  2-tuple, like [CBM, Thrtl] with only-CAT enalbed, when CDP is enable,
> +  the COS corresponds to a 3-tuple, like [Code_CBM, Data_CBM, Thrtl]. If
> +  neither CAT nor CDP is enabled, things would be easier, one COS
> +  corresponds to one Thrtl.

How many bits in Thrtl field?
Is it decided by the hardware type?

> +# References
> +
> +"INTEL® RESOURCE DIRECTOR TECHNOLOGY (INTEL® RDT) ALLOCATION FEATURES" 
> [Intel® 64 and IA-32 Architectures Software Developer Manuals, 
> vol3](http://www.intel.com/content/www/us/en/processors/architectures-software-developer-manuals.html)
> +

I checked the document. The CAT is in Chapter 17.17. However, there is
no description about the MBA? ;-)

Thanks,

Meng

-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania
http://www.cis.upenn.edu/~mengxu/

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel