Xen project Mailing List

Hi All!

On Fri, Sep 5, 2014 at 12:56 AM, Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx> wrote:

On Thu, 4 Sep 2014, Oleksandr Dmytryshyn wrote:
> Hi to all.
>
> I want to implement cpufreq driver in the next way:
> 1. Cpufreq governor will be implemented in the Xen
> 2. dom0 will only change cpu frequency and voltage of the physical cpus
>
> But there are some nuances:
> 1. dom0 driver should read an information about operation points
> (frequencies and voltages) and cpu supply source from the device tree for each
> physical cpu. In the omap processor case this driver suspects that
> those settings
> located in the /cpus/cpu@0/ node. But hypervisor creates an cpu node
> for each vcpu
> for kernel dom0 in the device tree and those information is lost in the dom0.
> 2. What about this case if we will have some physical cpus with different
> operation points (for example 2 cpus) and we give only one cpu for dom0?
>
> How should I transfer all information from the original cpu@xxxxxx@n nodes
> about all physical cpus to the kernel dom0 driver? Maybe an additional
> nodes should be created by the hypervisor in the device tree for dom0
> and named as pcpu@xxxxxxx@n?

If we do that, wouldn't we require changes to the core OMAP drivers or
cpu initialization code in Linux (to parse "pcpu" instead of "cpu"
nodes)?Â I don't expect they would be easy to upstream or maintain going
forward.

I am trying to think of an alternative, such as passing the real cpu
nodes to dom0 but then adding status = "disabled", but I am not sure
whether Linux checks the status for cpu nodes. In addition this scheme
wouldn't support the case where dom0 has more vcpus than pcpus on the
system. Granted it is not very common and might even be detrimental for
performances, but we should be able to support it.

In case where dom0 has more vcpus than pcpus on the

system, the dom0 kernel is the most bug-prone place for pcpu cpufreq governor. So I still believe that separate driver domain with direct 1:1 vcpu:pcpu mapping is the best place for cpufreq governor. But it also reasonable to run cpufreq governor as userspace daemon in dom0.

Also what do you think about PM QoS support? On bare metal cpufreq is tightly integrated with PM QoS and intensively cooperate in frequency scaling.

With best regards,

Ian, what do you think about this?

> Oleksandr Dmytryshyn | Product Engineering and Development
> GlobalLogic
> M +38.067.382.2525
> www.globallogic.com
>
> http://www.globallogic.com/email_disclaimer.txt
>
>
> On Tue, Sep 2, 2014 at 9:46 PM, Andrii Tseglytskyi
> <andrii.tseglytskyi@xxxxxxxxxxxxxxx> wrote:
> >
> > Hi Stefano,
> >
> > Thank you for explanation.
> > I think this requires more and deeper investigation, but for sure dom0
> > must be able to do this.
> > Let us investigate this.
> >
> > Thank you,
> >
> > Regards,
> > Andrii
> >
> > On Tue, Sep 2, 2014 at 9:39 PM, Stefano Stabellini
> > <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> > > On Tue, 2 Sep 2014, Andrii Tseglytskyi wrote:
> > >> On Tue, Sep 2, 2014 at 4:00 AM, Stefano Stabellini
> > >> <stefano.stabellini@xxxxxxxxxxxxx> wrote:
> > >> > On Fri, 29 Aug 2014, Andrii Tseglytskyi wrote:
> > >> >> Hi,
> > >> >>
> > >> >> Stefano, Ian,
> > >> >>
> > >> >> Could you please clarify the following point:
> > >> >>
> > >> >> I agree that decision about frequency change should be taken by Xen
> > >> >> hypervisor. But what about hardware frequency changing?
> > >> >> In general when frequency changed to bigger value (for example from 1
> > >> >> GHz to 1.5 GHz) for ARM kernels sequence looks like the following:
> > >> >>
> > >> >> 1) cpufreq governor decides that frequency should be changed. This
> > >> >> decision is taken after analysing of CPU performance data taking in
> > >> >> account governor policy.
> > >> >> 2) cpufreq governor asks cpufreq driver about new frequency.
> > >> >> 3) cpufreq driver compares current and target frequencies and asks
> > >> >> cpufreq regulator about voltage change.
> > >> >> 4) cpufreq regulator send i2c command to standalone microchip, which
> > >> >> is responsible for voltage changing.
> > >> >> 5) cpufreq driver asks clock framework about new frequency for CPU clock
> > >> >> 6) clock framework performs frequency sanity checks, taking in account
> > >> >> clock parents and clock divider settings, and call platform specific
> > >> >> "set_frequency" callback.
> > >> >> 7) platform specific callback performs proper HW registers
> > >> >> configuration for newly selected frequency
> > >> >>
> > >> >> Also there are some special cases - for example for OMAP5+ when
> > >> >> frequency is changed to 1.5 GHz+, two additional HW IPs should be
> > >> >> triggered (ABB and DCC, if someone is familiar with OMAP5+ )
> > >> >>
> > >> >> So, for generic ARM kernel we have 3 entities to change frequency:
> > >> >>
> > >> >> - cpufreq governor
> > >> >> - cpufreq driver
> > >> >> - cpufreq regulator
> > >> >>
> > >> >> + 2 additional IP for OMAP5+
> > >> >> - ABB
> > >> >> - DCC
> > >> >>
> > >> >> Taking in account all above, it looks like it would be better to
> > >> >> implement only Xen cpufreq governor. Xen will take a decision about
> > >> >> new frequency, and kernel dom0 will perform other steps. Dom0 contains
> > >> >> all generic and platform specific frameworks, needed for frequency
> > >> >> changing.
> > >> >>
> > >> >> What do you think ?
> > >> >
> > >> > Keep in mind that the architecture must be able to handle the case where
> > >> > dom0 has only 1 or 2 vcpus on a 4 or 8 cores system with multiple
> > >> > physical cpus.
> > >> > Could dom0 change the frequency of a physical core or a physical cpu is
> > >> > not even running on? If that is not a problem, because cpus and
> > >> > frequency changing are decoupled enough in Linux to allow it, then I am
> > >> > OK with it. But I suspect they are not.
> > >> >
> > >>
> > >> Not sure that I got your point correctly - dom0 will change frequency
> > >> on physical CPU.
> > >> And in case of OMAP - this changing affects on both ARM physical cpus
> > >> - changing is coupled.
> > >> In case of other ARM platforms - changing may be not coupled (I've
> > >> heard that Snapdragon can change cpu freqs independently on each
> > >> physical cpu)
> > >
> > > Let me explain with a concrete example.
> > >
> > > Let's suppose that the platform has 2 physical cpus, each cpu has 4
> > > cores.Â Let's also supposed that dom0 has only 2 vcpus, currently
> > > running on core0 and core1 of cpu0.
> > >
> > > In this case would dom0 be able to change the frequency of core3 of
> > > cpu1, given that is not even running on it?
> > > If it can be done without any hacks, then we can go ahead with this
> > > approach.
> >
> >
> >
> > --
> >
> > Andrii Tseglytskyi | Embedded Dev
> > GlobalLogic
> > www.globallogic.com
>

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel

Vitaly Chernooky |ÂSenior Developer - Product Engineering and Development

GlobalLogic

P +380.44.4929695 ext.1136 M +380.98.7920568 S cvv_2k

www.globallogic.com

http://www.globallogic.com/email_disclaimer.txt

Re: [Xen-devel] cpufreq implementation for OMAP under xen hypervisor.