Re: [Xen-devel] Broken PCI device passthrough, after XSA-302 fix?

[Marek Marczykowski-Górecki <marmarek@xxxxxxxxxxxxxxxxxxxxxx>]

On Mon, Jan 06, 2020 at 03:16:11PM +0100, Marek Marczykowski-Górecki wrote:
> On Mon, Jan 06, 2020 at 03:04:20PM +0100, Marek Marczykowski-Górecki wrote:
> > On Mon, Jan 06, 2020 at 12:18:31PM +0100, Jan Beulich wrote:
> > > On 04.01.2020 02:07, Marek Marczykowski-Górecki  wrote:
> > > > I have a multi-function PCI device, behind a PCI bridge, that normally
> > > > I assign to a single domain. But now it fails with:
> > > > 
> > > > (XEN) [VT-D]d14: 0000:04:00.0 owned by d0!<G><0>assign 0000:05:00.0 to 
> > > > dom14 failed (-22)
> > > 
> > > Is this on the 1st attempt, or after the device had already been
> > > assigned to some (same or other) guest? After quite a bit of
> > > staring at the code I can't seem to be able to spot a difference
> > > in behavior for the 1st attempt, but you not saying explicitly
> > > that it would only happen on subsequent ones makes me assume you
> > > run into the issue right away.
> > 
> > Yes, it was the first try.
> > 
> > > > This is Xen 4.8.5 + XSA patches. It started happening after some update
> > > > during last few months, not really sure which one.
> > > 
> > > Having a smaller window would of course help, as would ...
> > 
> > The working version was just before XSAs of 2019-10-31  (which include
> > XSA-302).
> > But at this point, I'm not sure if no other configuration changes were
> > made (see below).
> > 
> > > > I guess it is because quarantine feature, so initial ownership of
> > > > 0000:05:00.0 is different than the bridge it is connected to.
> > > > I'm not sure if relevant for this case, but I also set
> > > > pcidev->rdm_policy = LIBXL_RDM_RESERVE_POLICY_RELAXED.
> > > > 
> > > > Booting with iommu=no-quarantine helps. Note I do not use `xl
> > > > pci-assignable-add` command, only bind the device to the pciback driver
> > > > in dom0.
> > > 
> > > ... knowing whether behavior differs when using this preparatory
> > > step.
> > 
> > xl pci-assignable-add doesn't make a difference with XSA-306 applied.
> > But I've tried xl pci-assignable-remove with interesting result:
> > It succeeded for 0000:05:00.0 and 0000:05:00.2, but failed for
> > 0000:05:00.1 with this message:
> > 
> > (XEN) [VT-D]d0: 0000:05:00.1 owned by d32753!<G><0>deassign 0000:05:00.1
> > from dom32753 failed (-22)
> 
> And now, after this operation (failed -remove) I get the following error
> on domain start, even with LIBXL_RDM_RESERVE_POLICY_RELAXED properly
> set:
> 
> (XEN) [VT-D]d13: 0000:05:00.1 owned by d32753!<G><0>assign 0000:05:00.1 to 
> dom13 failed (-22)
> 
> I've tried doing -add and -remove in different order and every time it
> fails for 0000:05:00.1, but works for other functions.
> I don't see anything special about this function, compared to others.
> 
> I'll reboot the system and try again...

After fresh reboot:

1. xl debug-keys Q says 0000:05:00.* are assigned to dom0.
2. xl pci-assignable-add 0000:05:00.* (in order: .0, .1, .2)
3. domain start (with LIBXL_RDM_RESERVE_POLICY_RELAXED set) fails:

(XEN) [VT-D]d5: 0000:04:00.0 owned by d0!<G><0>assign 0000:05:00.2 to dom5 
failed (-22)

4. xl debug-keys Q says 0000:05:00.* are assigned to d32753
5. domain start (with LIBXL_RDM_RESERVE_POLICY_RELAXED set) fails:

(XEN) [VT-D]d7: 0000:05:00.2 owned by d32753!<G><0>assign 0000:05:00.2 to dom7 
failed (-22)

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

Attachment: signature.asc
Description: PGP signature

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel