Re: [Xen-devel] [PATCH] VT-d: improve RMRR validity checking

[Weidong Han <weidong.han@xxxxxxxxx>]

Alex Williamson wrote:
> On Wed, 2010-03-10 at 12:25 +0800, Weidong Han wrote:
>   
> > Alex Williamson wrote:
> >     
> > > On Wed, 2010-03-10 at 11:28 +0800, Weidong Han wrote:
> > >   
> > >       
> > > > Alex Williamson wrote:
> > > >     
> > > >         
> > > > > On Wed, 2010-03-10 at 10:40 +0800, Weidong Han wrote:
> > > > >   
> > > > >       
> > > > >           
> > > > > > Alex Williamson wrote:
> > > > > >     
> > > > > >         
> > > > > >             
> > > > > > > I have a system with what I consider to be a valid DRHD that's getting
> > > > > > > tripped up on this patch.  The problem is that the DRHD includes an
> > > > > > > IOAPIC scope, where the IOAPIC is not materialized on the PCI bus.  I
> > > > > > > think Xen is being overzealous in it's validity checking and that this
> > > > > > > is a valid configuration.  What do others think?  Are IOAPICs a
> > > > > > > special case that we can allow to be non-existent on the PCI bus?
> > > > > > >   
> > > > > > >       
> > > > > > >           
> > > > > > >               
> > > > > > Yes, IOAPIC can be not pci-discoverable. IOAPICs are only reported in 
> > > > > > the "Include_all" DRHD, and our patch won't check if the device is 
> > > > > > pci-discoverable or not for the "Include_all" DRHD. So I think the patch 
> > > > > > is no problem unless IOAPIC is not included in the "Include_all" DRHD. 
> > > > > > Can you post your boot logs?
> > > > > >     
> > > > > >         
> > > > > >             
> > > > > Weidong,
> > > > >
> > > > > That's a very subtle restriction, and I'm not sure how it works in
> > > > > practice.  If I have a multi-IOH system, each with VT-d hardware, each
> > > > > supporting interrupt remapping, each with one or more IOAPICs below
> > > > > them, how can interrupt remapping work if we can only associate an
> > > > > IOAPIC with the "include all" DRHD?  I'm confused.  Thanks,
> > > > >   
> > > > >       
> > > > >           
> > > > Each IOH will have one "include all" DRHD which reports IOAPICs for each 
> > > > IOH.
> > > >     
> > > >         
> > > Wouldn't that imply multiple PCI segments?  The configuration I'm
> > > looking at has multiple IOHs, all on the same PCI segment.  By my
> > > reading of the spec, we're only allowed to declare INCLUDE_PCI_ALL for
> > > one DRHD within the segment.  Am I incorrect?  Thanks,
> > >   
> > >       
> > Currently multiple PCI segments are not supported in Xen yet. So you 
> > encounter issue on multiple PCI segment system. We will support it after 
> > xen 4.0.
> >     
>
> Which is exactly why we have multiple IOHs on the *same* PCI segment on
> this system.  How is it possible to support multiple IOHs, all on the
> same PCI segment, each with VT-d hardware with interrupt remapping
> support, each with one or more IOAPICs below them given the current
> code?  We cannot list the IOAPICs only under the INCLUDE_PCI_ALL DRHD
> because that wouldn't provide the right information in the right place
> for interrupt remapping on the other DRHDs.  We cannot specify
> INCLUDE_PCI_ALL on all of the DRHDs because the spec indicates we can
> only have one INCLUDE_PCI_ALL DRHD per PCI segment (besides, we can't
> have PCI sub-hierarchy scopes specified on INCLUDE_PCI_ALL DRHDs, which
> means we'd have no way to associate PCI devices to a specific DRHD if
> they all set this flag).
>
> I'm inclined to believe the hardware actually works correctly if we
> associate an IOAPIC to a non-INCLUDE_PCI_ALL DRHD, but this validity
> checking code prevents Xen from even trying to use it.
>
> Alex
>   
This patch is no problem on our platform which has two IOHs, two 
IOAPICs. But there is only one DRHD, which is also INCLUDE_PCI_ALL DRHD. 
Can you post your Xen boot logs?
Regards,
Weidong

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel