[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [RFC PATCH] xen/amd-iommu: Add interrupt remapping quirk for ath11k


  • To: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Thu, 13 Mar 2025 17:07:58 +0100
  • Autocrypt: addr=jbeulich@xxxxxxxx; keydata= xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A nAuWpQkjM1ASeQwSHEeAWPgskBQL
  • Cc: xen-devel@xxxxxxxxxxxxxxxxxxxx, Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, Xenia Ragiadakou <xenia.ragiadakou@xxxxxxx>, Roger Pau Monné <roger.pau@xxxxxxxxxx>, Jason Andryuk <jason.andryuk@xxxxxxx>
  • Delivery-date: Thu, 13 Mar 2025 16:13:22 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 13.03.2025 17:02, Andrew Cooper wrote:
> On 13/03/2025 3:47 pm, Roger Pau Monné wrote:
>> On Thu, Mar 13, 2025 at 11:30:28AM -0400, Jason Andryuk wrote:
>>> On 2025-02-27 05:23, Roger Pau Monné wrote:
>>>> On Wed, Feb 26, 2025 at 04:11:25PM -0500, Jason Andryuk wrote:
>>>>> The ath11k device supports and tries to enable 32 MSIs.  Linux in PVH
>>>>> dom0 and HVM domU fails enabling 32 and falls back to just 1, so that is
>>>>> all that has been tested.
>>>> DYK why it fails to enable 32?
>>> In Linux msi_capability_init()
>>>
>>>         /* Reject multi-MSI early on irq domain enabled architectures */
>>>         if (nvec > 1 && !pci_msi_domain_supports(dev,
>>> MSI_FLAG_MULTI_PCI_MSI, ALLOW_LEGACY))
>>>                 return 1;
>>>
>>> MSI_FLAG_MULTI_PCI_MSI is only set for AMD and Intel interrupt remapping,
>>> and Xen PVH and HVM don't have either of those.  They are using "VECTOR", so
>>> this check fails.
>> Oh, interesting.  So classic PV MSI domain supports
>> MSI_FLAG_MULTI_PCI_MSI, even when no IOMMU is exposed there either.
>>
>> Thanks, so it's nothing specific to Xen, just how Linux works.
> 
> This is something which TGLX and I have discussed in the past.  It is a
> mistake for any x86 system to do MSI multi-message without an IOMMU.

Well, with PVH there always will be an IOMMU, just that Linux can't see
it. Even with PV it should be the hypervisor to determine whether multi-
message MSI is possible. Hence how the classic (non-pvops) kernel had
worked in this regard.

Jan

> MSI multi-message gets you a power-of-2, aligned, block of vectors, up
> to a maximum of 32, which must always target the same CPU.
> 
> The LAPIC prioritisation is on groups of 16, aligned, vectors.
> 
> If MSI has 16 or fewer vectors, then any interrupt causes all others to
> be blocked owing to LAPIC behaviour.
> 
> With 32 vectors, you can get two vectors (one from the first 16, one
> from the second 16) where the higher vector can interrupt the lower
> one.  And you pay 32 vectors for this.
> 
> With the IOMMU, every message gets a controllable CPU and controllable
> priority, because they come from the IRTE, not the device.
> 
> Removing Multi-MSI support makes vector allocation much easier because
> you you never need to allocate/move blocks.
> 
> ~Andrew




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.