Re: [Xen-devel] PCIe 2.0, VT-d and Intel 82576 enhancement for Xen SR-IOV

[Espen Skoglund <espen.skoglund@xxxxxxxxxxxxx>]

[Yu Zhao]
> Yes, using the master BDF can move current logic into Dom0 and makes
> hypervisor cleaner. And it does work for VT-d spec 1.2.

> But if VT-d spec 1.3 (or AMD/IBM/Sun IOMMU specs) says that the ARI
> device and the Virtual Function have their own remapping unit or
> something like this, rather than use their masters', how could we
> support it using the master BDF?

If this happens the dom0 kernel will detect it and pass a different
master BDF to the hypervisor.  This was the whole point of my comment;
the hypervisor should need not know what type of device function it is
dealing with.  The logic for handling this should if possible be kept
out of the hypervisor (and if these kind of changes came along you
would still need dom0 support for handling it anyway).

>                                    Things evolve fast, we would need
> to add another hypercall to enhance the master BDF one after it's in
> 3.4 -- it would be like when the device_add was added, the VT-d spec
> didn't have such requirement, but now we have to add device_add_ext
> because the compatibility requirement.

> Passing these device specific information down and doing the IOMMU
> specific work inside the hypervisor hereditarily come with current
> passthrough architecture. After choosing putting all IOMMU things
> (both high level remapping data structures and logics, and low level
> hardware drivers) into hypervisor, we lost the flexibility to split
> the matching up logic and move it back to the Dom0 kernel.

I don't buy this argument.  You seem to be indicating that the
mechanism for configuring a given setup can not be separated from the
mechanism which actually enforces that configuration.  This is not
true.  It's all a matter of finding the right abstraction for the
configuration interface.  Flexibility need not be sacrificed.  I guess
the main problem here is that there was never much thought put into
how to best express the interfaces and abstractions for dealing with
IOMMUs, and as newer generations of IOMMU and PCIe hardware came along
the lack of flexibility in the original abstractions has come back to
bite us.

        eSk


> Thanks,
> Yu

> On Thu, Mar 19, 2009 at 08:05:55PM +0800, Espen Skoglund wrote:
>> Why put all this logic into Xen itself?  Finding the matching DRHD
>> unit will often "just work" anyway by the way the DHRD scope matching
>> is performed.
>> 
>> That said, yes, I get it that it might sometimes be better to actually
>> map the VFs and ARI functions back to the PF before matching.
>> However, why not extend the current device_add function and hypercall
>> with a master BDF pointing back to the BDF "owning" the function?
>> This leaves all of the matching up logic out of the hypervisor.  A
>> zero or -1 master BDF could indicate that there is no owner (i.e., it
>> owns itself).
>> 
>>      eSk
>> 
>> 
>> 
>> [Yu Zhao]
>>> PCIe Alternative Routing-ID Interpretation (ARI) ECN defines the Extended
>>> Function -- a function whose function number is greater than 7 within an
>>> ARI Device. Intel VT-d spec 1.2 section 8.3.2 specifies that the Extended
>>> Function is under the scope of the same remapping unit as the traditional
>>> function. The hypervisor needs to know if a function is Extended Function
>>> so it can find proper DMAR for it.
>> 
>>> And section 8.3.3 specifies that the SR-IOV Virtual Function is under the
>>> scope of the same remapping unit as the Physical Function. The hypervisor
>>> also needs to know if a function is the Virtual Function and which Physical
>>> Function it's associated with for same reason.


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel