[Xen-devel] x86/AMD: Nested VM failed to boot L2 guest due to setting/clearing CR0.CD bit

[Suravee Suthikulanit <suravee.suthikulpanit@xxxxxxx>]

Hi All,

While I was testing nested VM on with latest Xen on AMD system, I am running 
into issue where
the L2 guest (Linux) seems to stuck right after loading the kernel. When using the 
"xl debug-keys d" to dump registers,
the L2 guest RIP always at the instruction which tries to write the CR0.CD bit. 
 Besides, once starting L2 guest and it
got stuck, L0 Dom0 becomes very slow until I kill the L2 guest.

After looking into the hvm code for handling CR0 (i.e. xen/arch/x86/hvm/hvm.c: 
hvm_set_cr0()),
I see that the code tries to issue local cache flush on all the cores when the 
L2 guest is
setting the CR0.CD bit. (Please see the code snippet below.)

        if ( (value & X86_CR0_CD) && !(value & X86_CR0_NW) )
        {
            /* Entering no fill cache mode. */
            spin_lock(&v->domain->arch.hvm_domain.uc_lock);
            v->arch.hvm_vcpu.cache_mode = NO_FILL_CACHE_MODE;

            if ( !v->domain->arch.hvm_domain.is_in_uc_mode )
            {
                /* Flush physical caches. */
---> HERE       on_each_cpu(local_flush_cache, NULL, 1);
                hvm_set_uc_mode(v, 1);
            }
            spin_unlock(&v->domain->arch.hvm_domain.uc_lock);
        }

When I try to comment out the line, the issue goes away.  Is this line 
necessary?
Why do we need to flush all the cpu cores when the CR0.CD bit only applies
to a particular core?

Thank you,

Suravee



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel