Re: [Xen-ia64-devel] [IPF-ia64] with Cset 10690, creating aVTImakexen0 hang

[Isaku Yamahata <yamahata@xxxxxxxxxxxxx>]

Hi SAKAI.

You misunderstood. There are two issues.

- one issues is related to stop_timer()
  stop_timer() prevents the message.
  "Ooops, pending guest timer before its due"
  This isn't very serious.

- another issues is related to TIMER_SLOP.
  This issue is that non-VTi domain hangs. This isn't fixed by stop_timer().
  As Anthony pointed out, adding TIMER_SLOP isn't complete fix.
  This issue is serious.

It seems to take a while for you to fix them.
Then, please back out your change at first.

Thanks.

On Tue, Jul 18, 2006 at 10:56:53PM +0900, Atsushi SAKAI wrote:
> Hi, Anthony and Yamahata
> 
> Thank you for your comments. and sorry for late.
> I know Anthony modification(stop_timer) fix the problem in empirically. 
> But, I am investigating the reason.
> 
> The problem is two timer function exists after defining the hlt_timer_fn.
> Two timer function confliction exists within the  vcpu_timer_expired => 
> vcpu_pend_timer => DomU => vcpu_set_itm or hyper_set_itm
> Because domain_itm keeps old value until vcpu_set_itm or hyper_set_itm is 
> called.
> For this reason, it has possibility that another VIRQ_ITC interrupt can call 
> during this window.
> To fix this problem, it should exclude VIRQ_ITC interrupt duing that time 
> window.
> 
> 
> and this problem is reproduced by using lmbench.(Thanks)
> 
> 
> Thanks
> Atsushi SAKAI
> 
> 
> 
> >Hi, Yamahata,
> >
> >If this domain depends on hlt_timer to wake it up,
> >The scenario you described can appear,
> >TIMER_SLOP can reduce this situation, but can't eliminate this situation.
> >Maybe we need another mechanism to wake up VCPU, when this VCPU
> >has been blocked for a certain time.
> >
> >Thanks,
> >Anthony
> >
> >
> >
> >>From: Isaku Yamahata 
> >>Hi SAKAI.
> >>
> >>The Anthony's modification is required.
> >>Without the modification, all physical CPUs end up in the idle loop and
> >>can't get out of it in spite of runnable vcpus existance because
> >>of TIMER_SLOP. This issue isn't related to VT-i domain.
> >>
> >>Thanks.
> >>
> >>On Wed, Jul 12, 2006 at 10:30:47PM +0800, Xu, Anthony wrote:
> >>> Hi, SAKAI
> >>> Forget wrong analysis in early email,
> >>>
> >>> In the phase of EFI boot, there are many IO operations, so dom0 is woken 
> >>> up
> >>by
> >>> VTIdomain in most time, while hlt_timer is still registered, this may 
> >>> cause
> >>more
> >>> timer interrupts injected to dom0 than before.
> >>>
> >>> Hope following modification help
> >>> #define TIMER_SLOP (50*1000) /* ns */
> >>>       set_timer(&v->arch.hlt_timer,
> >>vcpu_get_next_timer_ns(v)+TIMER_SLOP );
> >>>       do_sched_op_compat(SCHEDOP_block, 0);
> >>>       stop_timer(&v->arch.hlt_timer);
> >>>
> >>> Thanks,
> >>> Anthony
> >>>
> >>>
> >>> >-----Original Message-----
> >>> >From: xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx
> >>> >[mailto:xen-ia64-devel-bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Xu,
> >>Anthony
> >>> >Sent: 2006?7?12? 21:13
> >>> >To: Atsushi SAKAI; Alex Williamson; Zhang, Xiantao
> >>> >Cc: Isaku Yamahata; xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
> >>> >Subject: RE: [Xen-ia64-devel] [IPF-ia64] with Cset 10690,creating
> >>aVTImakexen0
> >>> >hang
> >>> >
> >>> >>From: Atsushi SAKAI
> >>> >>Sent: 2006?7?12? 19:05
> >>> >>My primary motivation is correct display of xenmon.py and xentop in
> >>BVT/CREDIT.
> >>> >>(N.B. SEDF displayed as same as x86, but BVT/CREDIT are not)
> >>> >>If only domU emulation is applied, it is a half way from my motivation.
> >>> >>Is dom0 dispatch another home work?
> >>> >>
> >>> >
> >>> >I suspect the slowness is not due to dom0 being scheduled out, but due to
> >>> >hlt_timer
> >>> >didn't work as expected.
> >>> >           set_timer(&v->arch.hlt_timer, vcpu_get_next_timer_ns(v));
> >>> >           do_sched_op_compat(SCHEDOP_block, 0);
> >>> >There is a time window between set_timer and dom0 being scheduled out, 
> >>> >and
> >>psr.i
> >>> >is 1. So if hlt_timer fires before do_sched_op_compat being called, dom0 
> >>> >will
> >>> >not be woken up by hlt_timer, and there is no timer interrupt for dom0 
> >>> >until
> >>> >domo
> >>> >is woken up ,yes, dom0 can be woken up by other external interrupts, but 
> >>> >not
> >>> >by
> >>> >timer interrupt. And since dom0 is involved in VTIdomain bootup, this may
> >>lead
> >>> >to slowness of VTIdomain bootup.
> >>> >
> >>> >Above is my analysis, there isn't any evidence.
> >>> >
> >>> >You can do experiment to check it.
> >>> >Change above code sequence as following, this may reduce the impact of 
> >>> >time
> >>> >window.
> >>> >
> >>> >set_timer(&v->arch.hlt_timer,
> >>> >cycle_to_ns(local_cpu_data->itm_delta)+NOW());
> >>> >do_sched_op_compat(SCHEDOP_block, 0);
> >>> >
> >>> >_______________________________________________
> >>> >Xen-ia64-devel mailing list
> >>> >Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
> >>> >http://lists.xensource.com/xen-ia64-devel
> >>>
> >>> _______________________________________________
> >>> Xen-ia64-devel mailing list
> >>> Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
> >>> http://lists.xensource.com/xen-ia64-devel
> >>
> >>--
> >>yamahata
> >
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> Xen-ia64-devel mailing list
> Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-ia64-devel

-- 
yamahata

_______________________________________________
Xen-ia64-devel mailing list
Xen-ia64-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-ia64-devel