Re: [RFC PATCH v1 00/10] Xen flamegraph (hypervisor stacktrace profile) support

[Demi Marie Obenour <demiobenour@xxxxxxxxx>]

On 7/25/25 11:06, Edwin Török wrote:
> Caveats:
>  * x86-only for now
>  * only tested on AMD EPYC 8124P
>  * Xen PMU support was broken to begin with on Xeon Silver 4514Y, so I
>  wasn't able to test there ('perf top' fails to parse samples). I'll
>  try to figure out what is wrong there separately
>  * for now I edit the release config in xen.spec to enable frame
>  pointers. Eventually it might be useful to have a 3rd build variant:
>  release-fp. Or teach Xen to produce/parse ORC or SFrame formats without
>  requiring frame pointers.

That would definitely be nice.

>  * perf produces raw hex addresses, and a python script is used to
>  post-process it and obtain symbols. Eventually perf should be updated
>  to do this processing itself (there was an old patch for Linux 3.12 by 
> Borislav Petkov)
>  * I've only tested capturing Dom0 stack traces. Linux doesn't support
>   guest stacktraces yet (it can only lookup the guest RIP)

What would be needed to fix this?  Capturing guest stacktraces from the host
or Xen seems like a really bad idea, but it might make sense to interrupt the
guest and allow it to provide a (strictly validated) stack trace for use by
the host.  This would need to be done asynchronously, as Linux is moving
towards generating stack traces outside of the NMI handler.

>  * the Linux patch will need to be forwarded ported to master before 
> submission
>  * All the caveats for using regular VPMU apply, except for the lack of
>   stacktraces, that is fixed here!
>     * Dom0 must run hard pinned on all host CPUs
>     * Watchdog must be disabled
>     * not security supported
>     * x86 only
>     * secureboot needs to be disabled

What would be needed to fix these limitations?  With them it isn't really
possible to do profiling on production systems, only on dedicated development
boxes.  That works great if you have a dev box and can create a realistic
workload with non-sensitive data, but less great if you have a problem that
you can't reproduce on a non-production system.  It's also not usable
for real-time monitoring of production environments.
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

Attachment: OpenPGP_0xB288B55FFF9C22C1.asc
Description: OpenPGP public key

Attachment: OpenPGP_signature.asc
Description: OpenPGP digital signature