Problem Description:
In the configuration
below, xen is booted in a bridge configuration
with firewalls disabled and forwarding enabled, which is what I want for layer
2 testing. The output below illustrates what happens on a ping from
the Virtual Router to the traffic generator. The results are the same
when the ping originates from the traffic generator. You can see the ICMP
request pass from vif1.0, across the eth2 bridge and finally across the peth2
nic i/f. You can then see the reply packet returned from the traffic
generator, pass through peth2 then arrive at eth2 at which point the bridge
appears to send an arp request to vif1.0 to resolve the 1.7.0.3 destination address on Virtual Router. This occurs
despite the fact the arp entry for 1.7.0.3 was manually added to the arp table
and exists when the request is made. Virtual Router sends its response to
the arp request, which shows up on the vif1.0 trace. The packet is never forwarded
to 1.7.0.3 and the ‘destination
unreachable’ message is sent by linux as a reply from 1.3.0.1 to the
traffic generator at 1.3.0.2. This really seems to be a configuration issue at the bridge, but I have sort of run out
of ideas at the moment.
Topology:
Traffic
|
Linux
Dom0
| Virt Router
Gen
|
|
| Intel 82576 Bridge Virtual
I/F |
|
|
1.3.0.2 ----- 1.3.0.1 -----eth2-----
1.7.0.1 ---------1.7.0.3
|
peth2
vif1.0 | Eth0/0
|
|
1.5.0.2 -------1.5.0.1 ----- eth3-----1.9.0.1
----------1.9.0.3
|
peth3
vif1.1 | Eth1/0
[root@localhost vr]# brctl show
bridge name bridge
id
STP enabled interfaces
eth2
8000.001b21242104
no
vif1.0
peth2
eth3
8000.001b21242105
no
vif1.1
peth3
[root@localhost vr]# arp -v
Address
HWtype HWaddress
Flags Mask
Iface
1.3.0.2
ether 00:10:94:00:00:02
C
eth2
192.168.102.65
ether 00:1C:F6:85:99:71
C
eth0
1.7.0.3
*
*
MP
vif1.0
1.5.0.2
*
* MP
eth3
1.9.0.3
*
*
MP
vif1.1
Ping
is sent from Virtual Router 1.7.0.3:
(vif1.0 trace)
[root@localhost vr]# tcpdump -i 2
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vif1.0, link-type EN10MB (Ethernet),
capture size 96 bytes
15:54:58.004454
15:55:08.004507
15:55:13.256470 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 0, length 80
15:55:13.261149 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:13.264651 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:14.261145 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:14.264040 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:15.002408 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 1, length 80
15:55:15.261237 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:15.263594 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:17.001942 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 2, length 80
15:55:17.011211 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:17.014329 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:18.005692
15:55:18.011224 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:18.013886 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:19.001560 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 3, length 80
15:55:19.011189 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:19.013990 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:21.000988 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 4, length 80
15:55:21.011187 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:21.013402 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:22.011245 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:22.013058 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:23.011214 arp who-has 1.7.0.3 tell 1.7.0.1
15:55:23.013193 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
(peth2 trace)
[root@localhost vr]# tcpdump -i 4
tcpdump: WARNING: peth2: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv
for full protocol decode
listening on peth2, link-type EN10MB (Ethernet),
capture size 96 bytes
15:55:13.256497 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 0, length 80
15:55:13.256785 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 0, length 80
15:55:15.002427 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 1, length 80
15:55:15.002665 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 1, length 80
15:55:16.261339 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:16.261347 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:17.001960 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 2, length 80
15:55:17.002207 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 2, length 80
15:55:18.256067 arp who-has 1.3.0.1 tell 1.3.0.2
15:55:18.256082 arp reply 1.3.0.1 is-at
00:1b:21:24:21:04 (oui Unknown)
15:55:19.001577 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 3, length 80
15:55:19.001834 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 3, length 80
15:55:20.011308 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:20.011316 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:21.001005 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 4, length 80
15:55:21.001237 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 4, length 80
15:55:24.011181 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:25.011241 arp who-has 1.3.0.2 tell 1.3.0.1
15:55:25.011438 arp reply 1.3.0.2 is-at
00:10:94:00:00:02 (oui Unknown)
15:55:25.012261 arp reply 1.3.0.2 is-at
00:10:94:00:00:02 (oui Unknown)
(eth2 trace)
[root@localhost vr]# tcpdump -i 5
tcpdump: verbose output suppressed, use -v or -vv
for full protocol decode
listening on eth2, link-type EN10MB (Ethernet),
capture size 96 bytes
15:55:13.256470 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 0, length 80
15:55:13.256785 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 0, length 80
15:55:13.264651 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:14.264040 arp reply 1.7.0.3 is-at 00:16:3e:08:1c:9c
(oui Unknown)
15:55:15.002408 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 1, length 80
15:55:15.002665 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 1, length 80
15:55:15.263594 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:16.261331 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:16.261346 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:17.001942 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 2, length 80
15:55:17.002207 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 2, length 80
15:55:17.014329 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:18.005692
15:55:18.013886 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:18.256067 arp who-has 1.3.0.1 tell 1.3.0.2
15:55:18.256079 arp reply 1.3.0.1 is-at
00:1b:21:24:21:04 (oui Unknown)
15:55:19.001560 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 3, length 80
15:55:19.001834 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 3, length 80
15:55:19.013990 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:20.011302 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:20.011315 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:21.000988 IP 1.7.0.3 > 1.3.0.2: ICMP echo
request, id 16, seq 4, length 80
15:55:21.001237 IP 1.3.0.2 > 1.7.0.3: ICMP echo
reply, id 16, seq 4, length 80
15:55:21.013402 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:22.013058 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:23.013193 arp reply 1.7.0.3 is-at
00:16:3e:08:1c:9c (oui Unknown)
15:55:24.011175 IP 1.3.0.1 > 1.3.0.2: ICMP host
1.7.0.3 unreachable, length 108
15:55:25.011235 arp who-has 1.3.0.2 tell 1.3.0.1
15:55:25.011438 arp reply 1.3.0.2 is-at
00:10:94:00:00:02 (oui Unknown)
15:55:25.012261 arp reply 1.3.0.2 is-at
00:10:94:00:00:02 (oui Unknown
[root@localhost vr]# ifconfig
eth0 Link
encap:Ethernet HWaddr 00:15:17:46:EF:CA
inet addr:192.168.102.94 Bcast:192.168.102.95 Mask:255.255.255.224
inet6 addr: fe80::215:17ff:fe46:efca/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:9075 errors:0 dropped:0 overruns:0 frame:0
TX packets:5180 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:1644721 (1.5 MiB) TX bytes:810984 (791.9 KiB)
Base address:0x2020 Memory:98820000-98840000
eth2 Link
encap:Ethernet HWaddr 00:1B:21:24:21:04
inet addr:1.3.0.1 Bcast:1.3.0.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe24:2104/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:429 errors:0 dropped:0 overruns:0 frame:0
TX packets:158 errors:0 dropped:0 overruns:0 carrier:0
collisions:0
txqueuelen:0
RX bytes:30865 (30.1 KiB) TX bytes:21598 (21.0 KiB)
eth3 Link encap:Ethernet
HWaddr 00:1B:21:24:21:05
inet addr:1.5.0.1 Bcast:1.5.0.255 Mask:255.255.255.0
inet6 addr: fe80::21b:21ff:fe24:2105/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:71 errors:0 dropped:0 overruns:0 frame:0
TX packets:57 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8500 (8.3 KiB) TX bytes:10139 (9.9 KiB)
lo Link
encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:12 errors:0 dropped:0 overruns:0 frame:0
TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:964 (964.0 b) TX bytes:964 (964.0 b)
peth2 Link
encap:Ethernet HWaddr 00:1B:21:24:21:04
inet6 addr: fe80::21b:21ff:fe24:2104/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:143 errors:0 dropped:0 overruns:0 frame:0
TX packets:299 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:12904 (12.6 KiB) TX bytes:43150 (42.1 KiB)
peth3 Link
encap:Ethernet HWaddr 00:1B:21:24:21:05
inet6 addr: fe80::21b:21ff:fe24:2105/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:31 errors:0 dropped:0 overruns:0 frame:0
TX packets:145 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2420 (2.3 KiB) TX bytes:25561 (24.9 KiB)
vif1.0 Link encap:Ethernet
HWaddr FE:FF:FF:FF:FF:FF
inet addr:1.7.0.1 Bcast:1.255.255.255 Mask:255.0.0.0
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1726 errors:0 dropped:0 overruns:0 frame:0
TX packets:183 errors:0 dropped:16 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:87769 (85.7 KiB) TX bytes:10749 (10.4 KiB)
vif1.1 Link encap:Ethernet
HWaddr FE:FF:FF:FF:FF:FF
inet addr:1.9.0.1 Bcast:1.9.0.255 Mask:255.255.255.0
inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1527 errors:0 dropped:0 overruns:0 frame:0
TX packets:29 errors:0 dropped:17 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:75672 (73.8 KiB) TX bytes:4214 (4.1 KiB)