Minimal performance gain with jumbo frames (MTU 9000) during VM migration

["Mario V. Guenzi" <jclark@xxxxxxxxxx>]

Hi everyone,

I'm trying to configure jumbo frames on two cluster nodes running Xen 4.17.
Both nodes are on Devuan 5, and the NICs are Intel Corporation Ethernet Controller 10-Gigabit X540-AT2.

At boot, I apply the following MTU settings:

post-up ifconfig eth05 mtu 9000 && ifconfig eth07 mtu 9000 && ifconfig bond1 mtu 9000

The two NICs are bonded with these settings:

bond-slaves eth05 eth07
bond-primary eth05
bond-mode 0
bond-miimon 100
bond-updelay 200
bond-downdelay 200

In rc.local I enable the following offload options:

ethtool -K eth05 rx on
ethtool -K eth05 tx on
ethtool -K eth05 sg on
ethtool -K eth05 tso on
ethtool -K eth05 gso on
ethtool -K eth05 gro off

Sysctl/kernel network tuning parameters:

net.core.somaxconn = 1000
net.core.netdev_max_backlog = 300000
net.core.rmem_max = 16777216
net.core.wmem_max = 16777216
net.ipv4.tcp_max_syn_backlog = 8096
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.ip_local_port_range = 10240 65535
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_wmem = 4096 65536 16777216
net.ipv4.tcp_mtu_probing = 1
net.ipv4.route.flush = 1
net.ipv4.tcp_timestamps = 0
net.ipv4.tcp_fin_timeout = 15
net.ipv4.tcp_sack = 0
net.ipv4.tcp_moderate_rcvbuf = 1
vm.vfs_cache_pressure = 50

The issue:
When I perform a VM live migration between the two nodes, the difference between MTU 1500 and 9000 is negligible—only about 2–3 seconds faster with jumbo frames enabled.

Does anyone know what I might be missing or doing wrong?
Any tips or insights would be greatly appreciated.

Thanks in advance and best regards

---

-- 
 
Mario Vittorio Guenzi
E-mail jclark@xxxxxxxxxx
Si vis pacem, para bellum