Re: [PATCH v3 4/5] evtchn: convert domain event lock to an r/w one

[Julien Grall <julien@xxxxxxx>]

Hi Jan,

On 14/12/2020 09:40, Jan Beulich wrote:
> On 11.12.2020 11:57, Julien Grall wrote:
> > On 11/12/2020 10:32, Jan Beulich wrote:
> > > On 09.12.2020 12:54, Julien Grall wrote:
> > > > On 23/11/2020 13:29, Jan Beulich wrote:
> > > > > @@ -620,7 +620,7 @@ int evtchn_close(struct domain *d1, int
> > > > >         long           rc = 0;
> > > > >     
> > > > >      again:
> > > > > -    spin_lock(&d1->event_lock);
> > > > > +    write_lock(&d1->event_lock);
> > > > >     
> > > > >         if ( !port_is_valid(d1, port1) )
> > > > >         {
> > > > > @@ -690,13 +690,11 @@ int evtchn_close(struct domain *d1, int
> > > > >                     BUG();
> > > > >     
> > > > >                 if ( d1 < d2 )
> > > > > -            {
> > > > > -                spin_lock(&d2->event_lock);
> > > > > -            }
> > > > > +                read_lock(&d2->event_lock);
> > > > This change made me realized that I don't quite understand how the
> > > > rwlock is meant to work for event_lock. I was actually expecting this to
> > > > be a write_lock() given there are state changed in the d2 events.
> > > Well, the protection needs to be against racing changes, i.e.
> > > parallel invocations of this same function, or evtchn_close().
> > > It is debatable whether evtchn_status() and
> > > domain_dump_evtchn_info() would better also be locked out
> > > (other read_lock() uses aren't applicable to interdomain
> > > channels).
> > >
> > > > Could you outline how a developper can find out whether he/she should
> > > > use read_lock or write_lock?
> > > I could try to, but it would again be a port type dependent
> > > model, just like for the per-channel locks.
> > It is quite important to have clear locking strategy (in particular
> > rwlock) so we can make correct decision when to use read_lock or write_lock.
> >
> > > So I'd like it to
> > > be clarified first whether you aren't instead indirectly
> > > asking for these to become write_lock()
> > Well, I don't understand why this is a read_lock() (even with your
> > previous explanation). I am not suggesting to switch to a write_lock(),
> > but instead asking for the reasoning behind the decision.
> So if what I've said in my previous reply isn't enough (including the
> argument towards using two write_lock() here), I'm struggling to
> figure what else to say. The primary goal is to exclude changes to
> the same ports. For this it is sufficient to hold just one of the two
> locks in writer mode, as the other (racing) one will acquire that
> same lock for at least reading. The question whether both need to use
> writer mode can only be decided when looking at the sites acquiring
> just one of the locks in reader mode (hence the reference to
> evtchn_status() and domain_dump_evtchn_info()) - if races with them
> are deemed to be a problem, switching to both-writers will be needed.
I had another look at the code based on your explanation. I don't think 
it is fine to allow evtchn_status() to be concurrently called with 
evtchn_close().
evtchn_close() contains the following code:

  chn2->state = ECS_UNBOUND;
  chn2->u.unbound.remote_domid = d1->domain_id;

Where chn2 is a event channel of the remote domain (d2). Your patch will 
only held the read lock for d2.
However evtchn_status() expects the event channel state to not change 
behind its back. This assumption doesn't hold for d2, and you could 
possibly end up to see the new value of chn2->state after the new 
chn2->u.unbound.remote_domid.
Thanksfully, it doesn't look like chn2->u.interdomain.remote_domain 
would be overwritten. Otherwise, this would be a straight dereference of 
an invalid pointer.
So I think, we need to held the write event lock for both domain.

Cheers,

--
Julien Grall