Re: [Xen-devel] pv_ops routing problem?

[Michael D Labriola <mlabriol@xxxxxxxx>]

xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/28/2010 09:07:20 AM:

> Michael D Labriola <mlabriol@xxxxxxxx> 
> Sent by: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> 
> 04/28/2010 09:07 AM
> 
> To
> 
> Michael D Labriola <mlabriol@xxxxxxxx>
> 
> cc
> 
> Jayson A Dyke <jdyke@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, 
> Jeremy Fitzhardinge <jeremy@xxxxxxxx>, 
xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> 
> Subject
> 
> Re: [Xen-devel] pv_ops routing problem?
> 
> xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/28/2010 06:40:32 AM:
> 
> > Michael D Labriola <mlabriol@xxxxxxxx> 
> > Sent by: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > 
> > 04/28/2010 06:40 AM
> > 
> > To
> > 
> > Jeremy Fitzhardinge <jeremy@xxxxxxxx>
> > 
> > cc
> > 
> > Jayson A Dyke <jdyke@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx, xen-
> > devel-bounces@xxxxxxxxxxxxxxxxxxx
> > 
> > Subject
> > 
> > Re: [Xen-devel] pv_ops routing problem?
> > 
> > xen-devel-bounces@xxxxxxxxxxxxxxxxxxx wrote on 04/27/2010 06:02:12 PM:
> > 
> > > Jeremy Fitzhardinge <jeremy@xxxxxxxx> 
> > > Sent by: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx
> > > 
> > > 04/27/2010 06:02 PM
> > > 
> > > To
> > > 
> > > Michael D Labriola <mlabriol@xxxxxxxx>
> > > 
> > > cc
> > > 
> > > Jayson A Dyke <jdyke@xxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxx
> > > 
> > > Subject
> > > 
> > > Re: [Xen-devel] pv_ops routing problem?
> > > 
> > > On 04/27/2010 02:22 PM, Michael D Labriola wrote:
> > > > Ok, I've got a problem.  My current setup is as follows:
> > > >
> > > > I have a pv_ops dom0 (xen/master from a few weeks ago) running 3 
> > domUs. 
> > > > Each domU has an eth0 interface and one or more VLAN interfaces.
> > > >
> > > > vm1, vm2, and vm3 are all domUs running with the same pv_ops 
kernel 
> as 
> > 
> > > > dom0.  vm1's eth0.107 interface routes through vm3 to get to vm2's 

> > > > eth0.1020 interface.
> > > >
> > > > dom0:
> > > >   eth0:     192.168.1.20/24
> > > >
> > > > vm1:
> > > >   eth0:     192.168.1.10/24
> > > >   eth0.107: 10.7.1.1/16
> > > >   gateway:  10.7.0.1
> > > >
> > > > vm2:
> > > >   eth0:      192.168.1.11/24
> > > >   eth0.1020: 10.20.1.1/16
> > > >   gateway:   10.20.0.1
> > > >
> > > > vm3:
> > > >   eth0:      192.168.1.12/24
> > > >   eth0.107:  10.7.0.1/16
> > > >   eth0.1020: 10.20.0.1/16
> > > >
> > > >
> > > > >From vm1, I can ping the gateway (10.7.0.1) and vm3's 1020 
> interface 
> > > > (10.20.0.1), but get no response if I try to ping vm2's 10.20.1.1.
> > > >
> > > > vm3 can ping both 10.7.1.1 and 10.20.1.1.
> > > >
> > > > Should be a pretty simple packet forwarding example, and it used 
to 
> > work 
> > > > using the old xen-linux 2.6.18 kernel we were using on our 
deployed 
> > > > systems.  But now it's broken...
> > > >
> > > > I have verified that the exact same setup running on 3 physical 
PCs 
> > > > bare-metal works w/out any problems.  I had the problem initially 
on 
> 
> > Xen 
> > > > 3.4.2, but just upgraded to 4.0.0 and have the exact same 
symptoms.
> > > > 
> > > 
> > > Have you tried turning off tx checksum offload?  (ethtool -K <dev> 
tx 
> > off)
> > > 
> > > Though for me that just fixes network access between dom0 and domU;
> > > external traffic normally works fine.
> > 
> > Jeremy, I haven't tried that yet.  I'll try it this morning, although 
I 
> > don't really expect that to fix the problem.  I'm also going to pull 
> from 
> > the xen/master branch again to see if that makes a difference.  My 
last 
> > 2.6.31.12 kernel was pulled from your tree a few weeks ago.
> > 
> > By the way, the packet forwarding is definitely happening on my router 

> > domU.  If I look at vm3's eth0 w/ wireshark, I can see the VLAN 
> > encapsulated ICMP echo request coming in on the 107 VLAN and I can see 

> > that vm3 is forwarding it out over its 1020 VLAN interface.  However, 
if 
> I 
> > look at my dom0's vif3.0 interface with wireshark I only see the VLAN 
> 107 
> > traffic...  no sign of the 1020 forwarded packet anywhere.  It's as if 

> > forwarded packets aren't finding their way onto dom0's vif3.0.  I've 
> never 
> > seen this before, and was under the impression that dom0's vif 
> interfaces 
> > _WERE_ the domU's interfaces just with a different name... 
> 
> Should the tx checksum offload only be disabled on dom0?  And does it 
have 
> to get disabled before xend starts?  I've tried disabling it on just 
dom0 
> and also on domUs with no difference.
> 
> Also, I updated to the current xen/master and it behaves the same.
> 
> I just tried adding 107 and 1020 VLAN interfaces to my dom0 (10.7.0.2 
and 
> 10.20.0.2) and it turns out that dom0 also cannot ping the VLAN 
interfaces 
> on any of the domUs...  not sure if that's a separate issue or if it's 
all 
> related...

Once I lowered and re-raised dom0's VLAN interfaces dom0 could ping all 
the domU's VLAN interfaces just fine.  Not sure why I had to do that, 
though...

Still can't ping from vm1 to vm2 using v3 as the default gateway, though.

I've also tried this now on xen/stable with the exact same results.

-Mike

PS - Jeremy, when I pulled from you this morning xen/stable was actually 
one commit ahead of xen/stable-2.6.32.x... not sure if that was 
intentional.  ;-)

---
Michael D Labriola
Electric Boat
mlabriol@xxxxxxxx
401-848-8871 (desk)
401-848-8513 (lab)
401-316-9844 (cell)



_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel