[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v1 00/28] x86: initial SVM AVIC support


  • To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • From: dmukhin@xxxxxxxx
  • Date: Fri, 20 Nov 2026 09:57:27 -0800
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 148.163.138.245) smtp.rcpttodomain=lists.xenproject.org smtp.mailfrom=ford.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=ford.com; dkim=pass (signature was verified) header.d=saarlouis.ford.com; dkim=pass (signature was verified) header.d=ford.com; arc=none (0)
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eOGMHtepyaAFGpeJr/ohGJkErq4VE39nOW4cyuYwbkk=; b=lKp0YEIzwuK+ADg2XhB2lPz0fQDprohVu10X+QHAJT8JINYaFH/KIIIj+1LNC4hmwWQWF6efVTNz5LClXWtgOlkWDehpOIzs2Xi3hFQMUqbaTO2qEtdpmJKlwRb7BQfeWRR+CMbS7jyzd4AEDH/ihOxsLJbnopTlNOFpyr/7J+gh50s6Q/dKqS5JkCwv4BucDE5+T1bku9WrO4ovfJ3tUDH/CH6ZEWTbnnoeaeOO0fPX4rqSg0KjL7qnOmDFviOy8d8UUZbNXPqPTzU+/qqGu7/jLeoDtaaLr0CgPaCOxQWiYT7lDhY9kT+VO584ah2WIgNNotmxITfsBxAlRtRNKQ==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=e5hdmMewe32JNgaxaHDLnlDL5m2efb7Xg0W5GLeIhzkAAtzqsyKCujr3KkV1++Vcsko1VY/y8ypLmZDnS73YfqLDWuPCbiuSWHwYoSP5Vi3dodwlWXyQciRO2evyCnl3mpzUJ+iQmEEaGAuQytQLdvNwObbPfkEm2PM7IoubfjE7ouJmlYyfSJ/jkKdBh90x7sNnC3EMd7dxsaU6PMDs5F7B9q/gNRUziX3nfAPetbh2Jkl1k9aUtQQ0WJ6T9dLjtyoCdVuNmBkFLlf4ZJMo8/CO5k6ZlgRZ+Ea4q1g5rfYqSg6TkZB+6jiaN0Mlhsngfw4jNopms/FNLW/66/QNpw==
  • Authentication-results: eu.smtp.expurgate.cloud; dkim=pass header.s=ppford header.d=ford.com header.i="@ford.com" header.h="Cc:Content-Transfer-Encoding:Content-Type:Date:From:In-Reply-To:Message-ID:MIME-Version:References:Subject:To"; dkim=fail header.s=selector2-azureford-onmicrosoft-com header.d=azureford.onmicrosoft.com header.i="@azureford.onmicrosoft.com"; dkim=pass header.s=ppserprodsaar header.d=saarlouis.ford.com header.i="@saarlouis.ford.com" header.h="Cc:Content-Transfer-Encoding:Content-Type:Date:From:In-Reply-To:Message-ID:MIME-Version:References:Subject:To"; dkim=pass header.s=ppfserpocford header.d=ford.com header.i="@ford.com" header.h="Cc:Content-Transfer-Encoding:Content-Type:Date:From:In-Reply-To:Message-ID:MIME-Version:References:Subject:To"
  • Cc: dmukhin@xxxxxxxx, xen-devel@xxxxxxxxxxxxxxxxxxxx, jason.andryuk@xxxxxxx, jbeulich@xxxxxxxx, roger@xxxxxxxxxxxxxx, teddy.astie@xxxxxxxxxx
  • Delivery-date: Fri, 20 Nov 2026 17:57:59 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>
  • Pser-m365-app: SER-APP

On Fri, Nov 20, 2026 at 03:47:15PM +0000, Andrew Cooper wrote:
> On 05/11/2026 1:33 am, dmukhin@xxxxxxxx wrote:
> > Patch Series
> > ============
> >
> > Basically the entire work is divided into three big series:
> >  (A) Current series: initial SVM AVIC [5]
> >  (B) XTF interrupt latency test [6]
> >  (C) XTF infra in public CI [5]
> >
> > For this AVIC series:
> >
> >  Patches 1, 5, 6, 9, 11-16 introduce various cleanups and fixups.
> >
> >  Patch 20 changes XEN_DOMCTL_debug_op implementation to allow
> >  more debug operations added (like enabling/disabling AVIC at
> >  run-time - used by the XTF test).
> >
> >  Patches 2, 3, 4, 7, 8, 10, 17, 18-25 represent AVIC implementation
> >  including performance counters and run-time controls via debug
> >  hypercall.
> >
> >  Patches 26-28 cover various documentation updates.
> 
> Sorry this reply has taken longer to get to than I had intended.  I have
> committed a couple of trivial patches, but I think the rest of the work
> is going to take a bit more complicated rearranging.
> 
> First of all, I think there are two areas where we want to take other
> pending work as prerequisite.
> 
> 1) Moving the SVM/SEV leaves into CPU Policy.  (Series of mine needing
> dusting off and re-posting.)
> 
> In patch 4, you introduce a new command line option.  I know Intel has
> one, but it is part of a bigger anti-pattern we're trying to remove. 
> Wit the SVM leaf in policy, `cpuid=no-avic` on the command line will
> work to hide the feature from Xen, and without much effort we can make
> `cpuid=avic` work as an opt-in until it's enabled by default.

I have parameterized MSR doorbell path vs vcpu_kick() via

  svm_avic=nodoorbell 

in the current series. It was pretty useful to run experiments.

So AVIC may still need its own command line switch.
What do you think?

> 
> Also my series removes the SVM_FEATURE_* constants (they'll be
> auto-generated now), which simplifies your earlier patches.
> 
> 2) Removing the vcpu_id <-> apic_id relation.  (Series of Alejandro's,
> or at least the first part of it, needing dusting off and re-posting.)
> 
> In patch 7, you encode the guest's APIC_ID as it's vCPU ID * 2.  Yes
> this is currently true, but we're desperately trying to break this bogus
> relation.  It prohibits providing a sane topology to guests, and is one
> of the hard limits on HVM_MAX_VCPUs being 128.
> 
> Alejandro did a series making topology selection work.  The first half
> of it at least gets into a position where the toolstack chooses the
> APIC_IDs and we've got a proper variable in Xen with which to derive
> everything else from.  Getting at least this sorted will give you a
> proper base to build the logical/physical table upon.
> 
> 3) (Bonus).  We need to take a decision on whether we're going to block
> writes to APIC_ID in xAPIC mode or not.  I sent an RFC patch.  If we do
> decide to block writes to APIC_ID in xAPIC mode, then it means the
> logical/physical table is static for the lifetime of the domain.
> 
> 
> Now for some feedback in no specific order
> 
> a) Please have one single patch adding all the architectural enumeration
> in one go (CPUID, VMCB, etc).  i.e. merge parts of patch 2, 3 and 7.  (I
> have your patch 7 rebased over the VMCB cleanup which was merged after
> posting).  I find this helpful when reviewing because it separates "what
> is needed for the hardware" from "what is just internal book-keeping".
> 
> b) In patch 4 you have a single static avic_apic_access_page[].  That
> was XSA-412 (3b5beaf49033cd).
> 
> It's actually worse on AMD.  On Intel, any writes targeting a
> non-register part of the backing page are discarded.  On AMD, they're
> deposited in the backing page.  So, this page *must* be allocated per
> domain, and not global.
> 
> c) AMD has extensions to the APIC spec.  AVIC even accelerates some of
> these, and you feed the accesses into the current vlapic
> infrastructure.   For current CPUs, at a minimum that means reverting
> changes to eLVTs, but I think more generally we should probably have a
> CPU generation check during init and refuse to turn on AVIC until a
> human has checked which new hardware features leak through.
> 
> d) One very fun bug with AVIC is that a misaligned xmm store can write
> to two different registers at once (ymm can hit 3, and zmm can hit 5). 
> The whole write gets deposited (contrary to Intel, which explicitly
> discards outside of the first register hit), but only one VMExit
> occurs.  This allows the guest to write reserved bits.  We need to
> confirm that nothing in Xen is going to malfunction (e.g. check for
> BUG_ON()'s/etc. which ought to be downgraded to domain_crash()/etc.)
> because of finding non-architectural state.
> 
> e) You've got a number of fixes for interrupt-related things.  I think
> we need to get those fixed first and with robust testing in place before
> trying to add AVIC in on top.  I'll see about dusting off an XTF test I
> have on the matter, but we really need to fix CR8 handling before
> building further.
> 
> f) I'm not overly convinced with making use of debugop like that.  For
> now, maybe defer it to the end of the series.  I have had several
> occasions to need extra hypercalls to make testing happen nicely, so I
> think there is room to make something, but I suspect it will want to be
> a little different in nature.
> 
> I'm sure there's stuff I've missed, but I think this a long enough email
> for now.
> 
> There's an outstanding question on patch 5.  If you can answer it, I can
> commit.
> 
> Thanks,
> 
> ~Andrew
> 



 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.