Re: [Xen-devel] Dom0 hypercall for adding and removing PCI devices

[Keir Fraser <keir.fraser@xxxxxxxxxxxxx>]

Exactly my thought.

 K.

On 24/7/08 09:43, "Tian, Kevin" <kevin.tian@xxxxxxxxx> wrote:

> Isn't enough to first switch VT-d page table, and then flush IOTLB?
> As long as RMRRs are kept same in two VT-d tables, and in any
> time a valid entry (either in IOTLB or by walking) can be found,
> above sequence seems complete.
> 
> Thanks,
> Kevin
> 
>> From: Keir Fraser
>> Sent: 2008年7月24日 16:38
>> 
>> Can the pagetable switch not be made atomic (i.e., so that the
>> RMRR regions
>> appear continuously available throughout)? I'd have thought
>> that would just
>> naturally happen.
>> 
>> If creating/destroying RMRR mappings is part of the switch
>> operation, you'd
>> have to destroy RMRR mappings in the dom0 table after the
>> switch, and create
>> RMRR mappings in the fallback table before the switch. Or just
>> have RMRR
>> mappings always mapped into all tables.
>> 
>> -- Keir
>> 
>> On 24/7/08 09:32, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>> 
>>> We found USB (has RMRR) is firstly assigned to dom0, but
>> pci_bus_probe()
>>> failed, then it was removed from dom0. The removing needs
>> switch to RMRR
>>> VT-d page table. At the same time, BIOS was using its RMRR.
>>> 
>>> Randy (Weidong)
>>> 
>>> Keir Fraser wrote:
>>>> If a device is assigned to a domain (in this case dom0) then that
>>>> domain's VT-d pagetables will contain the RMRR mappings for that
>>>> device. Hence BIOS can perform DMA in those RMRR-indicated regions
>>>> without swapping to and fro between domain tables and fallback RMRR
>>>> tables. The new fallback RMRR table would be just that -- a fallback
>>>> table used for any device not currently assigned to any domain (and
>>>> hence those devices should only have DMAs initiated by the BIOS, if
>>>> at all). 
>>>> 
>>>>  -- Keir
>>>> 
>>>> On 24/7/08 09:20, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>> 
>>>>> I have another concern, when BIOS is initiating DMA operation using
>>>>> RMRR, can we use RMRR VT-d page table to replace dom0 VT-d page
>>>>> table? Does it result in some DMA failures?
>>>>> 
>>>>> Randy (weidong)
>>>>> 
>>>>> Han, Weidong wrote:
>>>>>> Espen Skoglund wrote:
>>>>>>> [Keir Fraser]
>>>>>>>> On 23/7/08 10:26, "Han, Weidong" <weidong.han@xxxxxxxxx> wrote:
>>>>>>>>>> So this would be one extra VT-d pagetable, for the
>> whole system,
>>>>>>>>>> which would be the fallback location for RMRR mappings for
>>>>>>>>>> devices which are currently not assigned to any domain? Thus
>>>>>>>>>> allowing firmware to successfully initiate DMA operations on
>>>>>>>>>> those devices? Sounds sensible.
>>>>>>> 
>>>>>>> Well, the VT-d page table for RMRR devices need not contain the
>>>>>>> whole system memory---only those regions specified in the DMAR
>>>>>>> tables. 
>>>>>>> 
>>>>>>>>> Is it possible that idle_domain owns the RMRR VT-d page table?
>>>>>>> 
>>>>>>>> If that's a convenient place to stash it then why not?
>> Either way,
>>>>>>>> seems you're going to have it special-cased in the code as
>>>>>>>> fallback owner for unassigned devices. It's possible that having
>>>>>>>> it stashed in the idle domain will simply make the code more
>>>>>>>> confusing. I'm not sure though.
>>>>>>> 
>>>>>>> Right.  I don't see any particular good reason to
>> associate it with
>>>>>>> the idle domain.  As noted above, the page table need not even
>>>>>>> cover the whole memory, and it will never change after being set
>>>>>>> up at boot time.  If special case code is needed anyway, then one
>>>>>>> might as well install a custom VT-d page table.
>>>>>> 
>>>>>> What does "custom VT-d page table" mean?
>>>>>> 
>>>>>> Due to domain id is not the same with DID field in context, we can
>>>>>> reverse a DID for RMRR VT-d page table, it can avoid to associate
>>>>>> with idle domain.
>>>>>> 
>>>>>> Currently we reassign the device from dom0 to target domain when
>>>>>> assign a device, and return the device to dom0 when target domain
>>>>>> tears down. It's not correct due to some devices may be
>> not assigned
>>>>>> to any domain. Current device_assigned() also needs to be changed.
>>>>>> Maybe it needs more changes on VT-d.
>>>>>> 
>>>>>> I have some concerns, maybe I missed something. Why did
>> you use dom0
>>>>>> hypercall approach to replace original method? What's the main
>>>>>> benefit? I also wonder it's suitable to wrap pci_bus_probe()
>>>>>> function. 
>>>>>> 
>>>>>> Randy (Weidong)
>>>>>> 
>>>>>>> 
>>>>>>> If supported by hardware, the extra page tables can even
>> be skipped
>>>>>>> altogether and the device marked as having passthrough access.
>>>>>>> That would give the RMRR device complete access to system memory,
>>>>>>> though. 
>>>>>>> 
>>>>>>> eSk
>>> 
>> 
>> 
>> 
>> _______________________________________________
>> Xen-devel mailing list
>> Xen-devel@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-devel
>> 



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel