[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Xen in ARM environment networking overload issue
Hello guys, https://xen.markmail.org/message/3w4oqeu5z7ryfbsb?q=xen_add_phys_to_mach_entry&page=1 DOM0/DOMU kernels version 5.15.19 Two XENified guests. They are equal. They both have the same configuration. It is enough to have one DomU and Dom0. Part of DomU configuration. type = "pvh" memory = 1024 vcpus = 1 It is not necessary to repeat full custom board configuration with all the bridges. It is enough to connect DomU with some external machine or with Dom0 through the network. You should have a perf3 compiled package on both ends. on Dom0 or external host you issue command: perf3 -s & on DomU you issue the command: perf3 -c [Dom0 or external host ip address] -k 400K -b 0 After enough short time you will see in your DomU kernel messages or in your xen console something like [ 2385.999011] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003e0de -> mfn=0x00000000008031f2: pfn=0x000000000003e0de -> mfn=0x00000000008044a0 already exists [ 2355.968172] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003bfca -> mfn=0x000000000080319c: pfn=0x000000000003bfca -> mfn=0x0000000000803276 already exists [ 2323.002652] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003b80e -> mfn=0x000000000080447d: pfn=0x000000000003b80e -> mfn=0x00000000008032a7 already exists [ 2302.036336] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003e0de -> mfn=0x0000000000803105: pfn=0x000000000003e0de -> mfn=0x00000000008044a0 already exists [ 2273.758169] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003b80e -> mfn=0x00000000008033fc: pfn=0x000000000003b80e -> mfn=0x00000000008032a7 already exists [ 2252.254857] xen_add_phys_to_mach_entry: cannot add pfn=0x000000000003b80e -> mfn=0x00000000008032f0: pfn=0x000000000003b80e -> mfn=0x00000000008032a7 already exists You will have a lot of those messages. Involved files arch/arm/xen/p2m.c drivers/net/xen-netback/netback.c drivers/net/xen-netback/common.h This problem arrived in the p2m.c file in the xen_add_phys_to_mach_entry function This function adds a new mapping XEN page pfn to the DomU gfn. It does via red/black tree Xen netback adapter structure placed in the common.h file. It contains xenvif_queue structure. There are some involved members from this structure. struct xenvif_queue { ... struct page *mmap_pages[MAX_PENDING_REQS]; pending_ring_idx_t pending_cons; ... u16 pending_ring[MAX_PENDING_REQS]; ... struct page *pages_to_map[MAX_PENDING_REQS]; ... } All the pages are stored in the xenvif_queue->mmap_pages They are allocated by their indexes. Those ones are storedsequentially in the xenvif_queue->pending_ring. Pages allocation depends on xenvif_queue->pending_cons. This value rounded up to MAX_PENDING_REQS value. Pages allocated for the request cyclically. When an intensive network traffic is in progress especially when a packets flow density has been growing, sooner or later we will run into the case when we do not have enough free pages. MAX_PENDING_REQS in our case is 256 This case arrives in the netback.c xenvif_tx_action->gnttab_map_refs call. Main work is in the xenvif_tx_build_gops. xenvif_tx_action is issued in NAPI context. So we could say it is something like the interrupt bottom half. This message is produced when the issued pfn is presented in the red/black tree. It is produced unconditionally. In the above mentioned condition this output degrades the performance drastically. I may offer a patch which decreases the amount of messages. Regards, Oleg
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |