Re: [Xen-devel] kernel bootup slow issue on ovm3.1.1

["zhenzhong.duan" <zhenzhong.duan@xxxxxxxxxx>]

Ccing satish who first find this issue.

ä 2012-08-10 22:22, Jan Beulich åé:
> > > > On 10.08.12 at 06:40, "zhenzhong.duan"<zhenzhong.duan@xxxxxxxxxx>  wrote:
> > ä 2012-08-09 18:35, Jan Beulich åé:
> > > > > > On 09.08.12 at 11:42, "zhenzhong.duan"<zhenzhong.duan@xxxxxxxxxx>   wrote:
> > > > ä 2012-08-08 23:01, Jan Beulich åé:
> > > > > > > > On 08.08.12 at 11:48, "zhenzhong.duan"<zhenzhong.duan@xxxxxxxxxx>    wrote:
> > > > > > ä 2012-08-07 16:37, Jan Beulich åé:
> > > > > > Some spin at stop_machine after finish their job.
> > > > > And here you'd need to find out what they're waiting for,
> > > > > and what those CPUs are doing.
> > > > They are waiting the vcpu calling generic_set_all and those spin at
> > > > set_atomicity_lock.
> > > > In fact, all are waiting generic_set_all
> > > I think we're moving in circles - what is the vCPU currently
> > > generic_set_all() then doing?
> > Add some debug print, generic_set_all->prepare_set->write_cr0 took much
> > time,
> > all else are quick. set_atomicity_lock serialized this process between
> > cpus, make it worse.
> > One iteration:
> > MTRR: CPU 2
> > prepare_set: before read_cr0
> > prepare_set: before write_cr0 ------*block here*
> Yeah, that CR0 write disables the caches, and that's pretty
> expensive on EPT (not sure why NPT doesn't use/need the
> same hook) when the guest has any active MMIO regions:
> vmx_set_uc_mode(), when HAP is enabled, calls
> ept_change_entry_emt_with_range(), which is a walk through
> the entire guest page tables (i.e. scales with guest size, or, to
> be precise, with the highest populated GFN).
>
> Going back to your original mail, I wonder however why this
> gets done at all. You said it got there via
>
> mtrr_aps_init()
>   \->  set_mtrr()
>       \->  mtrr_work_handler()
>
> yet this isn't done unconditionally - see the comment before
> checking mtrr_aps_delayed_init. Can you find out where the
> obviously necessary call(s) to set_mtrr_aps_delayed_init()
> come(s) from?
At bootup stage, set_mtrr_aps_delayed_init is called by native_smp_prepare_cpus.
mtrr_aps_delayed_init is always set to ture for intel processor in upstream 
code.

> > > > >     Does
> > > > > your hardware support Pause-Loop-Exiting (or the AMD
> > > > > equivalent, don't recall their term right now)?
> > > > I have no access to serial line, could I get the info by a command?
> > > "xl dmesg" run early enough (i.e. before the log buffer wraps).
> > Below is xl dmesg result for your reference. thanks
> > ...
> > (XEN) VMX: Supported advanced features:
> > (XEN)  - APIC MMIO access virtualisation
> > (XEN)  - APIC TPR shadow
> > (XEN)  - Extended Page Tables (EPT)
> > (XEN)  - Virtual-Processor Identifiers (VPID)
> > (XEN)  - Virtual NMI
> > (XEN)  - MSR direct-access bitmap
> > (XEN)  - Unrestricted Guest
> I'm sorry, I had expected this to be printed here, but it isn't.
> Hence I can't tell for sure whether PLE is implemented there,
> but given how long it has been available it ought to be when
> "Unrestricted Guest" is there (which iirc got introduced much
> later).
From VMCS dump, looks PAUSE exiting is 0, PLE is 1.
(XEN) *** Control State ***
(XEN) PinBased=0000003f CPUBased=b6a065fe SecondaryExec=000004eb

zduan

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel