[Xen-devel] VT-d scalability issue

["Zhai, Edwin" <edwin.zhai@xxxxxxxxx>]

Keir,

I have found a VT-d scalability issue and want to some feed backs.

When I assign a pass-through NIC to a linux VM and increase the num of VMs, the
iperf throughput for each VM drops greatly. Say, start 8 VM running on a machine
with 8 physical cpus, start 8 iperf client to connect each of them, the final
result is only 60% of 1 VM.

Further investigation shows vcpu migration cause "cold" cache for pass-through
domain.

following code in vmx_do_resume try to invalidate orig processor's cache when   
                                      
14 migration if this domain has pass-through device and no support for wbinvd
vmexit.                                                                         
                                      
16 if ( has_arch_pdevs(v->domain) && !cpu_has_wbinvd_exiting )
{
    int cpu = v->arch.hvm_vmx.active_cpu;
    if ( cpu != -1 )
        on_selected_cpus(cpumask_of_cpu(cpu), wbinvd_ipi, NULL, 1,

}

So we want to pin vcpu to free processor for domains with pass-through device in
creation process, just like what we did for NUMA system.

What do you think of it? Or have other ideas?

Thanks,


-- 
best rgds,
edwin

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel