Re: Minimal performance gain with jumbo frames (MTU 9000) during VM migration

["Mario V. Guenzi" <jclark@xxxxxxxxxx>]

I apologize, I clicked reply instead of reply all.

Hello Cyrus, First of all, thanks for the reply and sorry for my english. 

I didn't measure the throughput on the ports. I simply scp'd between the two nodes of a 3 GB ISO and used netperf in this form: netperf -t TCP_STREAM -H xen2 -l 10 

obtaining this result:

MIGRATED TCP STREAM TEST from 0.0.0.0 (0.0.0.0) port 0 AF_INET to xen2 () port 0 AF_INET: demo Recv Send Send Socket Socket Message Elapsed Size Size Size Time Throughput bytes bytes bytes secs. 10^6 bits/sec 131072 65536 65536 10.03 486.86 

And the differences between the two MTUs are really minimal, we're talking 1-2 seconds. I expected the 10 GB cards to have significantly higher speeds.

Best regards.

Il 08/10/25 13:06, Cyrus ha scritto:

Good morning Mario, 

Usually Jumbo Frames will lead you to lower CPU usage for the same heavy workload, not more throughput.

What's the throughput you are seen on those ports?.

Regards,

CI

On Tue, Oct 7, 2025, 11:52 Mario V. Guenzi <jclark@xxxxxxxxxx> wrote:

Hi everyone,

I'm trying to configure jumbo frames on two cluster nodes running Xen 4.17.
Both nodes are on Devuan 5, and the NICs are Intel Corporation Ethernet Controller 10-Gigabit X540-AT2.

At boot, I apply the following MTU settings:

post-up ifconfig eth05 mtu 9000 && ifconfig eth07 mtu 9000 && ifconfig bond1 mtu 9000

The two NICs are bonded with these settings:

bond-slaves eth05 eth07
bond-primary eth05
bond-mode 0
bond-miimon 100
bond-updelay 200
bond-downdelay 200

In rc.local I enable the following offload options:

ethtool -K eth05 rx on
ethtool -K eth05 tx on
ethtool -K eth05 sg on
ethtool -K eth05 tso on
ethtool -K eth05 gso on
ethtool -K eth05 gro off

Sysctl/kernel network tuning parameters:

net.core.somaxconn = 1000
net.core.netdev_max_backlog = 300000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_max_syn_backlog = 8096
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_mtu_probing = 1
net.ipv4.route.flush = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_sack = 0
net.ipv4.tcp_moderate_rcvbuf = 1
vm.vfs_cache_pressure = 50

The issue:
When I perform a VM live migration between the two nodes, the difference between MTU 1500 and 9000 is negligible—only about 2–3 seconds faster with jumbo frames enabled.

Does anyone know what I might be missing or doing wrong?
Any tips or insights would be greatly appreciated.

Thanks in advance and best regards

---

-- 
 
Mario Vittorio Guenzi
E-mail jclark@xxxxxxxxxx
Si vis pacem, para bellum

-- 
 
Mario Vittorio Guenzi
E-mail jclark@xxxxxxxxxx
Si vis pacem, para bellum