Hi Sisyphean,
> On 9 Dec 2022, at 6:15 am, sisyphean
<sisyphean@zlw.email> wrote:
>
> Hi,
>
> I try to run XEN on my ARM board(Sorry, for some
commercial reasons, I can't tell you
> on which platform I run XEN) and add PCIE device
passthrough to DomU.But an error
> occurred while adding the PCIE device to SMMUv3.
PCI passthrough support is not fully upstream to Xen on
ARM. We have working
PCI passthrough branch that you can use to test it.
https://gitlab.com/xen-project/fusa/xen-integration/-/commits/integration/pci-passthrough
>
> Through reading the code and tracing debugging, the
error is found in the function
> arm_smmu_add_device, which will obtain and determine
whether the fwspec of the
> device to be added to SMMU exists.But for the XEN of
arm, the fwspec of the device is
> created and added by judging whether the iommu field
exists in the device node when
> XEN parses the device tree.However, the PCIE device
does not appear in the device tree,
> so there will be no fwspec for all PCIE devices. When
attempting to add a PCIE device to
> SMMU, a ENODEV error will be returned.
As of now Xen doesn’t support to add PCI device to IOMMU
on ARM.
>
> In addition, the code at
xen/drivers/passthrough/pci.c also verifies the above
view.
> For PCIE devices, pdev is alloc in function
pci_add_device by alloc_pdev.However,
> the function alloc_pdev does not create and add
fwspec to the PCIE device.Therefore,
> when function pci_add_device executes to
iommu_add_device,it will get the error
> return of ENODEV.
>
> How can I resolve the above errors?
If you want to test the PCI passthrough please follow
below steps.
Xen setup:
• A checkout of the “integration/pci-passthrough”
branch from the gitlab
https://gitlab.com/xen-project/fusa/xen-integration/-/commits/integration/pci-passthrough
• Pass iommu=yes and pci-passthrough=on to Xen
command line to enable PCI passthrough.
Linux Kernel setup:
• Some changes are required for the kernel to work
with PCI passthrough. First are some configuration
options, enable them in kernel config.
CONFIG_XEN=y
CONFIG_XEN_BACKEND=y
CONFIG_XEN_PCIDEV_BACKEND=y
• Then a patch needs to be applied for enabling the
pciback driver. Patch is attached in this email.
Using PCI passthrough:
• In order to pass a device to a guest, you first need
its PCI address(SBDF). You can either get it from a
bare-metal
Linux running on the platform or by having pciutils
installed (if you are using a yocto-based dom0 or have apt
available), which provides lspci.
For example, let's pass one ethernet interface to the
guest. Running lspci gives us this output (truncated) :
0000:00:00.0 Host bridge: Ampere Computing, LLC
Device e100
0000:00:01.0 PCI bridge: Ampere Computing, LLC
Device e101 (rev 04)
0000:01:00.0 Ethernet controller: Intel Corporation
Ethernet Controller X710/X557-AT 10GBASE-T (rev 01)
[...]
We will pass one of the ethernet from the PCI network
card : 0000:01:00.0 .
• Add the following line to the guest configuration
file :
pci = ['0000:01:00.0']
• Run the following command before starting the guest
:
xl pci-assignable-add 0000:01:00.0
• Start the guest. The network interface should appear
as 00:00.0 in the guest and be usable.
Please let me know if you need more info.
