Re: [Xen-users] vcpus and cpus, plus cpu affinity = ??

["Liang Yang" <multisyncfe991@xxxxxxxxxxx>]

Hi Mats,

You mentioned a very interesting feature of AMD CPU: The AMD setup for IO
is that each CPU has three HT-links, which can either be used for I/O or
interprocessor communication, so each CPU can have it's own set of
hardware that it can access.

Three questions (please bear with me if I misunderstood you):
1. How to bind a HT-link in a physcial CPU to I/O or IPC? Did you do this in 
BIOS?
2. Do you know if there is any CPU from Intel supports such kind of feature?

3. Each CPU has three HT-links. Is the HT-link you mentioned the same as the
Hyper Threads in Intel CPU? I heard most of CPUs only have two HT.

Thanks,

Liang

----- Original Message ----- 
From: "Petersson, Mats" <Mats.Petersson@xxxxxxx>
To: "Daniele Palumbo" <daniele@xxxxxxxxxxxx>; 
<xen-users@xxxxxxxxxxxxxxxxxxx>
Sent: Thursday, February 08, 2007 3:08 AM
Subject: RE: [Xen-users] vcpus and cpus, plus cpu affinity = ??




> -----Original Message-----
> From: xen-users-bounces@xxxxxxxxxxxxxxxxxxx
> [mailto:xen-users-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of
> Daniele Palumbo
> Sent: 08 February 2007 09:47
> To: xen-users@xxxxxxxxxxxxxxxxxxx
> Subject: [Xen-users] vcpus and cpus, plus cpu affinity = ??
>
> hi
>
> let's suppose i have something like that:
> ---
> Name                              ID  VCPU  CPU  State
> Time(s)  CPU Affinity
> Domain-0                           0     0    0   -b-
> 24842.7  any cpu
> Domain-0                           0     1    1   r--
> 4306.2  any cpu
> Domain-0                           0     2    2   -b-
> 5171.3  any cpu
> Domain-0                           0     3    3   -b-
> 2013.0  any cpu
> [snip]
> trip                              18     0    2   -b-
> 1189.8  any cpu
> uni147                            19     0    0   r--   112415.0  0
> uni147                            19     1    1   -b-   35259.4  1
> [snip]
> ---
>
> trip have 1 vcpu, cpus blank ("").
> uni147 have 2 vcpus, cpus = "0-3" (i have 4 opteron 844).
>
> which is the difference between dom0 and uni147, or better
> why one have "any
> cpu" and the other have "0" and "1" in cpu affinity?
>
> how xen can decide to allocate to one cpu or another?
> is supported something like "if cpu 0 will get full, move
> vcpu 0 of uni147 to
> cpu 3"?
> and for trip?
If you "hard-assign" the cpus in your config file, then that's the CPU's
you get.
If you don't, Xen just assings "at random".

Xen's Credit scheduler, which is what's in Xen by default since 3.0.2 (I
think), can, and will, move VCPU's between physical CPUs "for best
utilization of the CPU resources". There's no "automatic" move things
around between physical CPU's when you have hard-coded which CPU's you
want (at least not outside the set!).

As to what works best for any particular setup, that's a much harder to
answer question - and one that you'll probably be better placed to
answer than I am.
> am i wrong if i consider cpu0 the one that will get most of
> IO of the HW?
That depends on the hardware on your motherboard. The AMD setup for IO
is that each CPU has three HT-links, which can either be used for I/O or
interprocessor communication, so each CPU can have it's own set of
hardware that it can access. However, in practice, most motherboards
will attach most of the IO-devices to CPU0, since if you want to run the
system without a full set of CPU's, the first CPU is the one that you
have to use... But there are systems with two PCI-X controllers, one for
CPU0 and another for CPU1 or some such (in which case, you need to have
two CPU's to access all PCI-X slots).

That doesn't in itself mean that CPU0 is doing more work tho', as CPU1,
2, 3 can all access the hardware that is attached to CPU0, just that it
takes a fraction longer (but we're talking about PCI access with a
33/66/133MHz bus, compared to the 800MHz-1GHz of the HT-link, so the
latency is most likely not at all noticable - the overhead is roughly
40ns per hop, where CPU{1,3} will have 1 hop, CPU2 has two hops). Note
that a PCI access lasts at least 6-7 cycles in general (200ns+), and not
unusually longer than that.

> the cpu time of dom0 is affected by the usage of the domU for
> that real cpu?
Dom0's "usage" will be entirely based on what Dom0 is doing, but of
course, if DomU is sharing time with DomU, then the time used by DomU is
obviously not available to Dom0. Not sure what you're asking here...

> thanks
> Daniele
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users