RE: [PATCH v1 0/7] Implement support for external IPT monitoring

["Tian, Kevin" <kevin.tian@xxxxxxxxx>]

+Luwei, who developed PT for KVM and is the best one who can help
review VMX changes from Intel side. Please include him in future
post or discussion.

> -----Original Message-----
> From: Michał Leszczyński <michal.leszczynski@xxxxxxx>
> Sent: Wednesday, June 17, 2020 2:48 AM
> To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
> Cc: Xen-devel <xen-devel@xxxxxxxxxxxxxxxxxxxx>; Jan Beulich
> <jbeulich@xxxxxxxx>; Wei Liu <wl@xxxxxxx>; Roger Pau Monné
> <roger.pau@xxxxxxxxxx>; Nakajima, Jun <jun.nakajima@xxxxxxxxx>; Tian,
> Kevin <kevin.tian@xxxxxxxxx>; George Dunlap <george.dunlap@xxxxxxxxxx>;
> Ian Jackson <ian.jackson@xxxxxxxxxxxxx>; Julien Grall <julien@xxxxxxx>;
> Stefano Stabellini <sstabellini@xxxxxxxxxx>
> Subject: Re: [PATCH v1 0/7] Implement support for external IPT monitoring
> 
> ----- 16 cze 2020 o 20:17, Andrew Cooper andrew.cooper3@xxxxxxxxxx
> napisał(a):
> 
> > On 16/06/2020 16:16, Michał Leszczyński wrote:
> >> Intel Processor Trace is an architectural extension available in modern
> Intel
> >> family CPUs. It allows recording the detailed trace of activity while the
> >> processor executes the code. One might use the recorded trace to
> reconstruct
> >> the code flow. It means, to find out the executed code paths, determine
> >> branches taken, and so forth.
> >>
> >> The abovementioned feature is described in Intel(R) 64 and IA-32
> Architectures
> >> Software Developer's Manual Volume 3C: System Programming Guide,
> Part 3,
> >> Chapter 36: "Intel Processor Trace."
> >>
> >> This patch series implements an interface that Dom0 could use in order to
> enable
> >> IPT for particular vCPUs in DomU, allowing for external monitoring. Such a
> >> feature has numerous applications like malware monitoring, fuzzing, or
> >> performance testing.
> >
> > Hello,
> >
> > I'm very excited to see support like this appearing.  However, be aware
> > that we're currently in code freeze for the 4.14 release, so in-depth
> > reviews will probably be delayed somewhat due to our bug queue and
> > release activities.
> 
> Sure, take your time :)
> 
> 
> >
> > That said, I've had a very quick look through the series, and have a few
> > general questions first.
> >
> > AFAICT, this is strictly for external monitoring of the VM, not for the
> > VM to use itself?  If so, it shouldn't have the H tag here:
> >
> > XEN_CPUFEATURE(IPT,           5*32+25) /*H  Intel Processor Trace */
> >
> > because that exposes the feature to the guest, with the implication that
> > all other parts of the feature work as advertised.
> 
> Ok, I will remove the H tag.
> 
> 
> >
> >
> > Are there any restrictions on EPT being enabled in the first place?  I'm
> > not aware of any, and in principle we could use this functionality for
> > PV guests as well (using the CPL filter).  Therefore, I think it would
> > be helpful to not tie the functionality to HVM guests, even if that is
> > the only option enabled to start with.
> 
> I think at the moment it's not required to have EPT. This patch series doesn't
> use any translation feature flags, so the output address is always a machine
> physical address, regardless of context. I will check if it could be easily 
> used
> with PV.
> 
> 
> >
> > The buffer mapping and creation logic is fairly problematic.  Instead of
> > fighting with another opencoded example, take a look at the IOREQ
> > server's use of "acquire resource" which is a mapping interface which
> > supports allocating memory on behalf of the guest, outside of the guest
> > memory, for use by control tools.
> >
> > I think what this wants is a bit somewhere in domain_create to indicate
> > that external tracing is used for this domain (and allocate whatever
> > structures/buffers are necessary), acquire resource to map the buffers
> > themselves, and a domctl for any necessary runtime controls.
> >
> 
> I will check this out, this sounds like a good option as it would remove lots 
> of
> complexity from the existing ipt_enable domctl.
> 
> >
> > What semantics do you want for the buffer becoming full?  Given that
> > debugging/tracing is the goal, I presume "pause vcpu on full" is the
> > preferred behaviour, rather than drop packets on full?
> >
> 
> Right now this is a ring-style buffer and when it would become full it would
> simply wrap and override the old data.
> 
> >
> > When this subject was broached on xen-devel before, one issue was the
> > fact that all actions which are intercepted don't end up writing any
> > appropriate packets.  This is perhaps less of an issue for this example,
> > where the external agent can see VMExits in the trace, but it still
> > results in missing information.  (It is a major problem for PT within
> > the guest, and needs Xen's intercept/emulation framework being updated
> > to be PT-aware so it can fill in the same packets which hardware would
> > have done for equivalent actions.)
> 
> Ok, this sounds like a hard issue. Could you point out what could be the
> particular problematic cases? For instance, if something would alter EIP/RIP
> or CR3 then I belive it would still be recorded in PT trace (i.e. these 
> values will
> be logged on VM entry).
> 
> >
> >
> > Thanks,
> >
> > ~Andrew
> 
> 
> Best regards,
> Michał Leszczyński
> CERT Polska