Re: [Xen-devel] Questions about Using Perf at Dom0

[Sanghyun Hong <shhong@xxxxxxx>]

I really appreciate for your answers. Last but not the least, I want to make it 
more clear:

> The hypervisor will provide dom0 with a raw sample (guest's, dom0's or
> hypervisor's) and then it's the job of dom0 kernel to properly tag and
> format it and make it available to the userland (i.e. perf itself).

Does this mean if I run the perf command on Dom0 while other domains are 
running, we can collect the performance counter values without distinguishing 
each domain, right?

> The tool will then look at the tag and display the event as belonging to
> host (dom0, really) or guest. This is supported for KVM, with 'perf kvm'
> commands.
> 
> So if you can run perf kvm commands then you may be able to
> differentiate dom0's and guest's samples (you will also see hypervisor
> samples but they won't get resolved to symbols). I am just not sure per
> kvm will run on a non-KVM host. I had a private copy where it did but
> this was based on a fairly old version of Linux.
> 
> (Also, stack profiling is not supported at all).


I wonder if you can share the private copy of your code. I’m looking into the 
source code of the patches, and it seems there’s a way to do it. Even if your 
codes are outdated, I think I can manage to port it. I will promise once I 
finish the porting, I will share my adjustment of your code as well.

Best,
Sanghyun.


> On Aug 29, 2016, at 5:24 PM, Boris Ostrovsky <boris.ostrovsky@xxxxxxxxxx> 
> wrote:
> 
> On 08/29/2016 05:08 PM, Sanghyun Hong wrote:
>>> Yes, this will allow the hypervisor to collect samples from multiple
>>> guests. However, the tool (perf) probably won't be able to properly
>>> process these samples. But you can try.
>> 
>> I understand, thus, I applied the patches and set
>> the /pmu_mode/ to *all*. However, I’m really curious what you mean by
>> the tool (*perf*) probably won’t be able to properly process these
>> samples. Is there any things I have to have in mind while collecting
>> the counters, or will it have incorrect values of the counters?
> 
> The hypervisor will provide dom0 with a raw sample (guest's, dom0's or
> hypervisor's) and then it's the job of dom0 kernel to properly tag and
> format it and make it available to the userland (i.e. perf itself). The
> tool will then look at the tag and display the event as belonging to
> host (dom0, really) or guest. This is supported for KVM, with 'perf kvm'
> commands.
> 
> So if you can run perf kvm commands then you may be able to
> differentiate dom0's and guest's samples (you will also see hypervisor
> samples but they won't get resolved to symbols). I am just not sure per
> kvm will run on a non-KVM host. I had a private copy where it did but
> this was based on a fairly old version of Linux.
> 
> (Also, stack profiling is not supported at all).
> 
>> 
>>> You will want to run dom0 on all physical processors (i.e. no
>>> dom0_max_vcpus boot option) and pin all VCPUs for both dom0 and the
>>> guest.
>> 
>> Sure, thanks! My machine has four physical cores, and I’ve set
>> the /dom0_max_vcpus=4/. I think it will have the same effect in
>> removing the /dom0_max_vcpus/ option.
> 
> Yes.
> 
> -boris
> 
>> 
>> Best,
>> Sanghyun.
>> 
>> 
>>> On Aug 29, 2016, at 4:59 PM, Boris Ostrovsky
>>> <boris.ostrovsky@xxxxxxxxxx <mailto:boris.ostrovsky@xxxxxxxxxx>> wrote:
>>> 
>>> On 08/29/2016 02:42 PM, Sanghyun Hong wrote:
>>>> Hi Boris,
>>>> 
>>>> I’ve found the
>>>> documentations(https://github.com/torvalds/linux/blob/master/Documentation/ABI/testing/sysfs-hypervisor-pmu)
>>>> in the kernel source code, and it seems like if we change the mode
>>>> from *self* to *all* then we can collect counters for all the domains,
>>>> right?
>>> 
>>> Yes, this will allow the hypervisor to collect samples from multiple
>>> guests. However, the tool (perf) probably won't be able to properly
>>> process these samples. But you can try.
>>> 
>>> You will want to run dom0 on all physical processors (i.e. no
>>> dom0_max_vcpus boot option) and pin all VCPUs for both dom0 and the
>>> guest.
>>> 
>>> 
>>> -boris
>>> 
>>>> 
>>>> All the best,
>>>> Sanghyun.
>>>> 
>>>> 
>>>>> On Aug 29, 2016, at 12:36 PM, Boris Ostrovsky
>>>>> <boris.ostrovsky@xxxxxxxxxx <mailto:boris.ostrovsky@xxxxxxxxxx>
>>>>> <mailto:boris.ostrovsky@xxxxxxxxxx>> wrote:
>>>>> 
>>>>> On 08/29/2016 09:18 AM, Sanghyun Hong wrote:
>>>>>> Dear Xen-Devel Community:
>>>>>> 
>>>>>> I’m a grad student working on measuring performance counters at the
>>>>>> Xen domains. I read this
>>>>>> thread(https://wiki.xenproject.org/wiki/Xen_Profiling:_oprofile_and_perf)
>>>>>> in
>>>>>> web, and it says using Linux perf command will let us collecting
>>>>>> performance counters in both dom0 and domU. Does it mean that we can
>>>>>> collect both of them at once if we run perf command on the dom0? (If
>>>>>> not, does it mean we can collect counters for each domain separately
>>>>>> once we run the perf command in each domain?
>>>>> 
>>>>> Profiling both guest and dom0 (and the hypervisor) requires changes to
>>>>> perf and those are not there yet.
>>>>> 
>>>>> But you can run perf in each guest (including dom0) separately. Make
>>>>> sure you have vpmu=true boot option.
>>>>> 
>>>>> -boris
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>> 
>>> 
>> 
> 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel