[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-devel] Interesting observation with network event notification and batching




On 2013-6-17 18:35, Wei Liu wrote:
On Mon, Jun 17, 2013 at 10:38:33AM +0100, Ian Campbell wrote:
On Sun, 2013-06-16 at 10:54 +0100, Wei Liu wrote:
Konrad, IIRC you once mentioned you discovered something with event
notification, what's that?
They were bizzare. I naively expected some form of # of physical NIC
interrupts to be around the same as the VIF or less. And I figured
that the amount of interrupts would be constant irregardless of the
size of the packets. In other words #packets == #interrupts.

It could be that the frontend notifies the backend for every packet it
sends. This is not desirable and I don't expect the ring to behave that
way.
It is probably worth checking that things are working how we think they
should. i.e. that netback's calls to RING_FINAL_CHECK_FOR_.. and
netfront's calls to RING_PUSH_..._AND_CHECK_NOTIFY are placed at
suitable points to maximise batching.

Is the RING_FINAL_CHECK_FOR_REQUESTS inside the xen_netbk_tx_build_gops
loop right? This would push the req_event pointer to just after the last
request, meaning the net request enqueued by the frontend would cause a
notification -- even though the backend is actually still continuing to
process requests and would have picked up that packet without further
notification. n this case there is a fair bit of work left in the
backend for this iteration i.e. plenty of opportunity for the frontend
to queue more requests.

The comments in ring.h say:
  *  These macros will set the req_event/rsp_event field to trigger a
  *  notification on the very next message that is enqueued. If you want to
  *  create batches of work (i.e., only receive a notification after several
  *  messages have been enqueued) then you will need to create a customised
  *  version of the FINAL_CHECK macro in your own code, which sets the event
  *  field appropriately.

Perhaps we want to just use RING_HAS_UNCONSUMED_REQUESTS in that loop
(and other similar loops) and add a FINAL check at the very end?

But it was odd and I didn't go deeper in it to figure out what
was happening. And also to figure out if for the VIF we could
do something of #packets != #interrupts.  And hopefully some
mechanism to adjust so that the amount of interrupts would
be lesser per packets (hand waving here).
I'm trying to do this now.
What scheme do you have in mind?
Basically the one you mentioned above.

Playing with various event pointers now.

Did you collect data of how much requests netback processes when req_event is updated in RING_FINAL_CHECK_FOR_REQUESTS? I assume this value is pretty small from your test result. How about not updating req_event every time when there is no unconsumed request in RING_FINAL_CHECK_FOR_REQUESTS?

Thanks
Annie


Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.