Re: Re; [Xen-users] Ethernet has Alzheimers

[Gareth Bult <gareth@xxxxxxxxxxxxx>]

Wait, it gets better, on 10.0.0.12;

arp -d 10.0.0.1

tcpdump shows;

23:14:14.014725 arp who-has 10.0.0.1 tell 10.0.0.10
23:14:14.014797 arp reply 10.0.0.1 is-at fe:ff:ff:ff:ff:ff

Eeeek!

--
Managing Director, Encryptec Limited
Tel: 0845 25 77033, Mob: 07853 305393, Int: 00 44 1443205756
Email: gareth@xxxxxxxxxxxxx 
Statements made are at all times subject to Encryptec's Terms and Conditions of 
Business, which are available upon request.

----- Original Message -----
From: "Gareth Bult" <gareth@xxxxxxxxxxxxx>
To: "Matthew Crocker" <mcrocker@xxxxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Monday, January 7, 2008 11:11:04 PM (GMT) Europe/London
Subject: Re: Re; [Xen-users] Ethernet has Alzheimers

Ok, I've managed to pin it down and you are quite right - it's ARP.

Now the question is, how do I fix it.

here's what I have

Dom0 :: 10.0.0.1
DomU :: 10.0.0.12

Both machines work fine for 40 mins .. then;

DomU reports Dom0 unreachable.
Sure enough ping 10.0.0.1 gives no response.
However, ping 10.0.0.12 from Dom0 responds fine.
A one-way ping!

arp -na on Dom0 reports as expected.
arp -na on the broken DomU shows;
? (10.0.0.1) at FE:FF:FF:FF:FF:FF [ether] on eth0

It's picking up FE:EE ... instead of the desired MAC address ?!

How can it do this ?!

On Dom0:
eth0      Link encap:Ethernet  HWaddr 00:15:C5:5D:C0:BE  
          inet addr:10.0.0.1  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::215:c5ff:fe5d:c0be/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:205397 errors:0 dropped:0 overruns:0 frame:0
          TX packets:413848 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:41267633 (39.3 MB)  TX bytes:95050228 (90.6 MB)

On DomU:
eth0      Link encap:Ethernet  HWaddr 00:00:10:00:00:0C  
          inet addr:10.0.0.12  Bcast:10.0.0.255  Mask:255.255.255.0
          inet6 addr: fe80::200:10ff:fe00:c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6297 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11662 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1157351 (1.1 MB)  TX bytes:907972 (886.6 KB)

Help!




--
Managing Director, Encryptec Limited
Tel: 0845 25 77033, Mob: 07853 305393, Int: 00 44 1443205756
Email: gareth@xxxxxxxxxxxxx 
Statements made are at all times subject to Encryptec's Terms and Conditions of 
Business, which are available upon request.

----- Original Message -----
From: "Matthew Crocker" <mcrocker@xxxxxxxxxxx>
To: "Gareth Bult" <gareth@xxxxxxxxxxxxx>
Cc: xen-users@xxxxxxxxxxxxxxxxxxx
Sent: Monday, January 7, 2008 10:00:40 PM (GMT) Europe/London
Subject: Re: Re; [Xen-users] Ethernet has Alzheimers

>> Is the machine answering ARP replies?
>
> Honestly, I don't know .. the machine tends to lock up for other  
> reasons when it dies hence it's not easy to track ..
>
>> Does the upstream router have the IP & MAC in its ARP table?
>> Does the upstream switch have the MAC in its mac-address-table?
>> Assigned to the correct port?
>
>> Sounds to me like an ARP timeout problem.
>
> This occurs between DomU's and Dom0 in addition to external  
> addresses ... so I don't think it's linked to anything outside of  
> Xen .. I've experienced the same problem on 4 different machines,  
> all different HW config .. so again I think faulty HW is out.
>
> For what it's worth;
>
> I'm using Ubuntu Gutsy (7.10) with the stock Xen 3.1 kernel all on  
> AMD64 and Intel/Xeon machines all running 64 bit kernels and distros.
>
> All machines are using bridging with two physical ethernet ports.
> All DomU's are running two matching virtual ports.
> I'm using IPTABLES (firehol) fairly heavily for port filtering.

Bridging is Layer2,  IP is Layer 3, you are having a problem at layer  
3 so you need to look to make sure your layer 2 stuff is working  
properly.

If Xen is bridging only then you won't really have visibility into the  
Layer 3 problem from Dom0.  You could look at the bridging config and  
see if it knows about the MAC address properly in the switch.   At  
some point upstream from the Xen hardware you have another Layer 3  
device,  most likely a router.  You need to get into that router and  
see if it has the IP -> MAC entry in its ARP table.  If it doesn't  
have it then there is your problem.  Something is stopping the DomU  
from answering the ARP queries from the router.  The route loses track  
of the MAC address and can no longer send Ethernet frames to your  
DomU.  If your router does have the ARP entry then I would look into  
your switches and see if they are dropping the MAC address from their  
table.






_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users