Re: [win-pv-devel] Advice on evtchn interrupt handling

[Paul Durrant <Paul.Durrant@xxxxxxxxxx>]

> -----Original Message-----
> From: RafaÅ WojdyÅa [mailto:omeg@xxxxxxxxxxxxxxxxxxxxxx]
> Sent: 10 December 2015 12:56
> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
> 
> On 2015-12-10 13:27, Paul Durrant wrote:
> >> -----Original Message-----
> >> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-devel-
> >> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
> >> Sent: 09 December 2015 14:06
> >> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> >> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
> >>
> >> On 12/09/2015 02:02 PM, Paul Durrant wrote:
> >>>> -----Original Message-----
> >>>> From: RafaÅ WojdyÅa [mailto:omeg@xxxxxxxxxxxxxxxxxxxxxx]
> >>>> Sent: 09 December 2015 11:08
> >>>> To: Paul Durrant; win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> >>>> Subject: Re: [win-pv-devel] Advice on evtchn interrupt handling
> >>>>
> >>>> On 2015-12-09 11:44, Paul Durrant wrote:
> >>>>>> -----Original Message-----
> >>>>>> From: win-pv-devel-bounces@xxxxxxxxxxxxxxxxxxxx [mailto:win-pv-
> >> devel-
> >>>>>> bounces@xxxxxxxxxxxxxxxxxxxx] On Behalf Of Rafal Wojdyla
> >>>>>> Sent: 09 December 2015 10:19
> >>>>>> To: win-pv-devel@xxxxxxxxxxxxxxxxxxxx
> >>>>>> Subject: [win-pv-devel] Advice on evtchn interrupt handling
> >>>>>>
> >>>>>> Hi,
> >>>>>>
> >>>>>> I found an issue in my evtchn handling in xeniface that I'm not sure
> how
> >>>>>> to properly fix. Namely, the interrupt callback that's assigned for a
> >>>>>> particular channel can be called before its queued DPC (that signals
> the
> >>>>>> caller-provided event) finishes. In such case the interrupt callback
> >>>>>> fails to insert the DPC again and an event is lost, potentially causing
> >>>>>> a vchan "deadlock" down the way.
> >>>>>>
> >>>>>
> >>>>> Hi Rafal,
> >>>>>
> >>>>> That shouldn't be the case. Windows de-queues a DPC before running
> >> for
> >>>> precisely this kind of reason. I.e. just because your DPC is running, it
> >> should
> >>>> not stop another from being queued. Are you definitely seeing a failure
> to
> >>>> queue even when no DPC is actually queued? What you may actually
> be
> >>>> getting caught by is evtchn auto-masking. If you have enabled auto-
> >> masking
> >>>> then you need to explicitly unmask at the end of your DPC. (The
> >> transmitter
> >>>> and receiver ring DPCs in XENVIF are a good example of this).
> >>>>>
> >>>>> Cheers,
> >>>>>
> >>>>>   Paul
> >>>>>
> >>>> Here's the relevant bit:
> >>>>
> >>
> http://xenbits.xen.org/gitweb/?p=pvdrivers/win/xeniface.git;a=blob;f=src/x
> >>>>
> >>
> eniface/ioctl_evtchn.c;h=9c5af19e00a91a08f68da0c630709cbaa6e8413d;hb=
> >>>> HEAD#l66
> >>>>
> >>>> I check the result of KeInsertQueueDpc() and I definitely see it fail
> >>>> sometimes, even if the HVM only has 1 vcpu. My DPC unmasks the
> >> channel
> >>>> after setting an event so this shouldn't be the reason.
> >>>>
> >>>
> >>> The article at https://www.osronline.com/article.cfm?article=529 is
> worth
> >> reading. The comment labelled "DPC Reentrancy" points out that DPCs
> are
> >> de-queued before being executed. So really you should not see a
> queueing
> >> failure unless either then unmasking is not working correctly or there's a
> bug
> >> in Windows itself.
> >>
> >> Yeah, I've read that before and was wondering whether I'm missing
> >> something that would explain the behavior I see.
> >
> > Actually, looking more closely at the code today I do see a problem...
> >
> > You have a DPC per CPU so, if you have two separate event channels
> bound to the same CPU you're gonna hit problems. E.g. an event comes in on
> the first channel so you schedule a DPC to handle it. Now, an event comes in
> on the second channel and you can't schedule the DPC because itâs still
> queued for the first event. You need a DPC per channel.
> >
> >   Paul
> >
> Ha, that's exactly what I found as well. I implemented a similar change
> yesterday and everything seemed fine during my tests. Thanks for the
> confirmation - I think you can apply the patch.
> 

Cool. Thanks :-)

  Paul

> >>
> >>> Are you running on a Xen with FIFO events or is it old enough to only be
> 2L?
> >>>
> >>>   Paul
> >>
> >> We're using 4.4.3 right now.
> >>
> >>>
> >>>>>> Any advice on how to approach this? The interrupt callback runs at
> >>>>>> HIGH_LEVEL which is an issue when it comes to synchronization with
> >>>> some
> >>>>>> other code that could take care of such "pending" events.
> >>>>>>
> >>>>>> Also I wonder if the kernel differentiates DPCs only by their KDPC
> >>>>>> address, or also by arguments passed to KeInsertQueueDpc()?
> >>>>>>
> 
> --
> RafaÅ WojdyÅa
> Qubes Tools for Windows developer
> https://www.qubes-os.org/
_______________________________________________
win-pv-devel mailing list
win-pv-devel@xxxxxxxxxxxxxxxxxxxx
http://lists.xenproject.org/cgi-bin/mailman/listinfo/win-pv-devel