Re: [Xen-devel] FW: VT-d async invalidation for Device-TLB.

["Jan Beulich" <JBeulich@xxxxxxxx>]

>>> On 13.06.15 at 16:44, <quan.xu@xxxxxxxxx> wrote:
>> On 12.06.15 at 14:47, <JBeulich@xxxxxxxx> wrote:
>> >>> On 12.06.15 at 04:40, <quan.xu@xxxxxxxxx> wrote:
>> >> > >>> On 10.06.15 at 16:05, <JBeulich@xxxxxxxx> wrote:
>> >> >>> On 03.06.15 at 09:49, <quan.xu@xxxxxxxxx> wrote:
>> >> >     For Context Invalidation and IOTLB invalidation without
>> >> > Device-TLB invalidation, Invalidation Queue flushes synchronous
>> >> > invalidation as before(This is a tradeoff and the cost of interrupt is
>> overhead).
>> >>
>> >> DMAR_OPERATION_TIMEOUT being 1s, are you saying that you're not
>> >> intending to replace the current spinning for the non-ATS case?
>> >
>> > Yes, we are not intending to replace the current spinning for the
>> > non-ATS case.
>> 
>> I'm not really happy about that.
> 
> Jan, could you share more about what you expect? We can enable it step by 
> step.
> Do you want to replace the current spinning for all of queued invalidation? 

Yes.

>> >> Considering that expiring these loops results in panic()s, I would
>> >> expect
>> > these to
>> >> become asynchronous _and_ contained to the affected VM alongside the
>> >> ATS induced changed behavior. You talking of overhead - can you quantify
>> that?
>> >>
>> >
>> > I tested it by a Myri-10G Dual-Protocol NIC, which is an ATS device.
>> > for an invalidation:
>> >  By sync way, it takes about 1.4 ms.
>> >  By async way, it takes about 4.3 ms.
>> 
>> What's the theory on why this is? After all, it shouldn't matter how the
>> completion of the invalidation gets signaled.
>>
> 
> Let me introduce more about how I get these test data.
> 
> For sync way, I get the time by NOW() macro, when I update Queue Tail 
> Register  (qinval_update_qtail()).
> Then get time by NOW() macro in current spinning when it has wrote the value 
> in the Status Data field to the address specified in the Status Address.
> The difference of these 2 value is the time of an sync invalidation.
> 
> 
> For async way, as the previous email, I have introduced the IF bit of 
> Invalidation Wait Descriptor. 
> (IF: Indicate the invalidation wait descriptor completion by generating an 
> invalidation completion event per the programing of the Invalidation 
> Completion Event Registers.)
> I have implemented an interrupt for invalidation completion event. 
> Also I get the time by NOW() macro, when I update Queue Tail Register (by 
> qinval_update_qtail()).
> Then get time by NOW() macro in invalidation completion event handler.
> The difference of these 2 value is the time of an async invalidation.

Okay, thanks for the explanation. As this includes the time it takes to
deliver and (partly) handle the interrupt, the difference is of course
within what one would expect (and also of what would seem acceptable).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel