Re: [Xen-API] guest network not working

[James Bulpin <James.Bulpin@xxxxxxxxxxxxx>]

My guess would be that it’s something to do with your guest having three interfaces (VIFs) on the same L2 network each with its own IP address in the same subnet. In my experience this kind of thing can lead to asymmetric routing/flows and ultimately to connectivity problems. Do you really mean to have three VIFs on the same network (the bond)?

 

James

 

From: xen-api-bounces@xxxxxxxxxxxxx [mailto:xen-api-bounces@xxxxxxxxxxxxx] On Behalf Of Carlos Reategui
Sent: 27 March 2013 18:49
To: xen-api@xxxxxxxxxxxxx
Subject: Re: [Xen-API] guest network not working

 

Here is some additional info.  Hopefully someone has some ideas.  I am thinking of trying the same setup with Debian Wheezy and also with XCP 1.6 to see if they behave the same:

 

# brctl show

bridge name     bridge id               STP enabled     interfaces

xapi0           8000.feffffffffff       no              vif0.0

                                                        vif7.0

                                                        vif9.0

xapi1           8000.e0db5516cfea       no              bond0

                                                        vif7.1

                                                        vif7.2

                                                        vif9.1

                                                        vif9.2

                                                        vif9.3

xenbr0          8000.e0db5516cfe8       no

xenbr1          8000.e0db5516cfe9       no

 

Dmesg:

[ 5582.806362] block tda: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5585.700640] block tda: sector-size: 512/512+0 capacity: 98304 discard: 0+0 flush: 0x0

[ 5586.663288] block tdb: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5587.222896] device vif6.0 entered promiscuous mode

[ 5587.227398] ADDRCONF(NETDEV_UP): vif6.0: link is not ready

[ 5587.507293] device vif6.1 entered promiscuous mode

[ 5587.511651] ADDRCONF(NETDEV_UP): vif6.1: link is not ready

[ 5587.803385] device vif6.2 entered promiscuous mode

[ 5587.807914] ADDRCONF(NETDEV_UP): vif6.2: link is not ready

[ 5589.157283] xen-blkback:ring-ref 8, event-channel 8, protocol 2 (x86_32-abi)

[ 5589.180779] xen-blkback:ring-ref 9, event-channel 9, protocol 2 (x86_32-abi)

[ 5589.201527] ADDRCONF(NETDEV_CHANGE): vif6.0: link becomes ready

[ 5589.201644] xapi0: port 2(vif6.0) entering forwarding state

[ 5589.201653] xapi0: port 2(vif6.0) entering forwarding state

[ 5589.210394] ADDRCONF(NETDEV_CHANGE): vif6.1: link becomes ready

[ 5589.210466] xapi1: port 2(vif6.1) entering forwarding state

[ 5589.210473] xapi1: port 2(vif6.1) entering forwarding state

[ 5589.218240] ADDRCONF(NETDEV_CHANGE): vif6.2: link becomes ready

[ 5589.218309] xapi1: port 3(vif6.2) entering forwarding state

[ 5589.218317] xapi1: port 3(vif6.2) entering forwarding state

[ 5598.212089] vif6.0: no IPv6 routers present

[ 5598.228086] vif6.1: no IPv6 routers present

[ 5598.228097] vif6.2: no IPv6 routers present

[ 5604.541317] xapi1: port 3(vif6.2) entering forwarding state

[ 5604.542059] device vif6.2 left promiscuous mode

[ 5604.542066] xapi1: port 3(vif6.2) entering disabled state

[ 5604.560787] xapi1: port 2(vif6.1) entering forwarding state

[ 5604.561314] device vif6.1 left promiscuous mode

[ 5604.561320] xapi1: port 2(vif6.1) entering disabled state

[ 5604.585109] xapi0: port 2(vif6.0) entering forwarding state

[ 5604.585515] device vif6.0 left promiscuous mode

[ 5604.585521] xapi0: port 2(vif6.0) entering disabled state

[ 5612.464696] block tda: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5614.807058] block tda: sector-size: 512/512+0 capacity: 98304 discard: 0+0 flush: 0x0

[ 5615.364312] block tdb: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5615.931075] device vif7.0 entered promiscuous mode

[ 5615.936000] ADDRCONF(NETDEV_UP): vif7.0: link is not ready

[ 5616.260102] device vif7.1 entered promiscuous mode

[ 5616.264585] ADDRCONF(NETDEV_UP): vif7.1: link is not ready

[ 5616.553472] device vif7.2 entered promiscuous mode

[ 5616.557941] ADDRCONF(NETDEV_UP): vif7.2: link is not ready

[ 5617.757138] xen-blkback:ring-ref 8, event-channel 8, protocol 2 (x86_32-abi)

[ 5617.785974] xen-blkback:ring-ref 9, event-channel 9, protocol 2 (x86_32-abi)

[ 5617.803397] ADDRCONF(NETDEV_CHANGE): vif7.0: link becomes ready

[ 5617.803502] xapi0: port 2(vif7.0) entering forwarding state

[ 5617.803520] xapi0: port 2(vif7.0) entering forwarding state

[ 5617.812314] ADDRCONF(NETDEV_CHANGE): vif7.1: link becomes ready

[ 5617.812388] xapi1: port 2(vif7.1) entering forwarding state

[ 5617.812396] xapi1: port 2(vif7.1) entering forwarding state

[ 5617.824975] ADDRCONF(NETDEV_CHANGE): vif7.2: link becomes ready

[ 5617.825043] xapi1: port 3(vif7.2) entering forwarding state

[ 5617.825050] xapi1: port 3(vif7.2) entering forwarding state

[ 5626.819648] vif7.0: no IPv6 routers present

[ 5626.827641] vif7.1: no IPv6 routers present

[ 5626.835633] vif7.2: no IPv6 routers present

[ 5647.942053] block tdc: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5650.549122] block tdc: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5651.110855] device vif8.0 entered promiscuous mode

[ 5651.115292] ADDRCONF(NETDEV_UP): vif8.0: link is not ready

[ 5651.398721] device vif8.1 entered promiscuous mode

[ 5651.403084] ADDRCONF(NETDEV_UP): vif8.1: link is not ready

[ 5651.697835] device vif8.2 entered promiscuous mode

[ 5651.702217] ADDRCONF(NETDEV_UP): vif8.2: link is not ready

[ 5651.990136] device vif8.3 entered promiscuous mode

[ 5651.994460] ADDRCONF(NETDEV_UP): vif8.3: link is not ready

[ 5652.802786] xen-blkback:ring-ref 8, event-channel 8, protocol 2 (x86_32-abi)

[ 5652.815915] xen-blkback:ring-ref 9, event-channel 9, protocol 2 (x86_32-abi)

[ 5652.930115] ADDRCONF(NETDEV_CHANGE): vif8.0: link becomes ready

[ 5652.930228] xapi0: port 3(vif8.0) entering forwarding state

[ 5652.930236] xapi0: port 3(vif8.0) entering forwarding state

[ 5652.939356] ADDRCONF(NETDEV_CHANGE): vif8.1: link becomes ready

[ 5652.939425] xapi1: port 4(vif8.1) entering forwarding state

[ 5652.939432] xapi1: port 4(vif8.1) entering forwarding state

[ 5652.946208] ADDRCONF(NETDEV_CHANGE): vif8.2: link becomes ready

[ 5652.946278] xapi1: port 5(vif8.2) entering forwarding state

[ 5652.946285] xapi1: port 5(vif8.2) entering forwarding state

[ 5652.957053] ADDRCONF(NETDEV_CHANGE): vif8.3: link becomes ready

[ 5652.957133] xapi1: port 6(vif8.3) entering forwarding state

[ 5652.957141] xapi1: port 6(vif8.3) entering forwarding state

[ 5661.939100] vif8.0: no IPv6 routers present

[ 5661.955097] vif8.1: no IPv6 routers present

[ 5661.955102] vif8.2: no IPv6 routers present

[ 5661.971094] vif8.3: no IPv6 routers present

[ 5667.639746] xapi1: port 6(vif8.3) entering forwarding state

[ 5667.640703] device vif8.3 left promiscuous mode

[ 5667.640710] xapi1: port 6(vif8.3) entering disabled state

[ 5667.660768] xapi1: port 5(vif8.2) entering forwarding state

[ 5667.661302] device vif8.2 left promiscuous mode

[ 5667.661308] xapi1: port 5(vif8.2) entering disabled state

[ 5667.685481] xapi1: port 4(vif8.1) entering forwarding state

[ 5667.686064] device vif8.1 left promiscuous mode

[ 5667.686070] xapi1: port 4(vif8.1) entering disabled state

[ 5667.708785] xapi0: port 3(vif8.0) entering forwarding state

[ 5667.708880] device vif8.0 left promiscuous mode

[ 5667.708885] xapi0: port 3(vif8.0) entering disabled state

[ 5674.740573] block tdc: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5677.403182] block tdc: sector-size: 512/512+0 capacity: 4194304 discard: 0+0 flush: 0x0

[ 5678.003996] device vif9.0 entered promiscuous mode

[ 5678.008405] ADDRCONF(NETDEV_UP): vif9.0: link is not ready

[ 5678.300580] device vif9.1 entered promiscuous mode

[ 5678.304986] ADDRCONF(NETDEV_UP): vif9.1: link is not ready

[ 5678.595727] device vif9.2 entered promiscuous mode

[ 5678.600230] ADDRCONF(NETDEV_UP): vif9.2: link is not ready

[ 5678.896861] device vif9.3 entered promiscuous mode

[ 5678.901276] ADDRCONF(NETDEV_UP): vif9.3: link is not ready

[ 5679.704222] xen-blkback:ring-ref 8, event-channel 8, protocol 2 (x86_32-abi)

[ 5679.717656] xen-blkback:ring-ref 9, event-channel 9, protocol 2 (x86_32-abi)

[ 5679.749436] ADDRCONF(NETDEV_CHANGE): vif9.0: link becomes ready

[ 5679.749543] xapi0: port 3(vif9.0) entering forwarding state

[ 5679.749552] xapi0: port 3(vif9.0) entering forwarding state

[ 5679.752986] ADDRCONF(NETDEV_CHANGE): vif9.1: link becomes ready

[ 5679.753060] xapi1: port 4(vif9.1) entering forwarding state

[ 5679.753067] xapi1: port 4(vif9.1) entering forwarding state

[ 5679.768976] ADDRCONF(NETDEV_CHANGE): vif9.2: link becomes ready

[ 5679.769045] xapi1: port 5(vif9.2) entering forwarding state

[ 5679.769053] xapi1: port 5(vif9.2) entering forwarding state

[ 5679.787610] ADDRCONF(NETDEV_CHANGE): vif9.3: link becomes ready

[ 5679.787680] xapi1: port 6(vif9.3) entering forwarding state

[ 5679.787688] xapi1: port 6(vif9.3) entering forwarding state

[ 5688.762675] vif9.0: no IPv6 routers present

[ 5688.770661] vif9.1: no IPv6 routers present

[ 5688.786657] vif9.2: no IPv6 routers present

[ 5688.802655] vif9.3: no IPv6 routers present

 

 

 

 

On Tue, Mar 26, 2013 at 8:44 PM, Carlos Reategui <creategui@xxxxxxxxx> wrote:

I only have 2 nics and they are bonded using xe.  This bond is both my guest and management network.  Dom0 has no network issues.  DomUs can not connect out:

 

Dom0 is running ubuntu 12.04 + xcp-xapi:

# dpkg -l | grep xcp

ii  xcp-eliloader                    0.1-4                        XenAPI's bootloader for EL-based guests

ii  xcp-fe                           0.5.2-3                      Fork-and-exec daemon for xapi

ii  xcp-guest-templates              0.1-3                        Guest template generator for XCP

ii  xcp-networkd                     1.3.2-5ubuntu0.1             Xen Cloud Platform - network configuration daemon

ii  xcp-squeezed                     1.3.2-5ubuntu0.1             Xen Cloud Platform - memory ballooning daemon

ii  xcp-storage-managers             0.1.1-2ubuntu1               storage backends for XCP

ii  xcp-v6d                          1.3.2-5ubuntu0.1             Xen Cloud Platform - feature daemon

ii  xcp-vncterm                      0.1-2                        Provides VNC service for XCP guest VMs

ii  xcp-xapi                         1.3.2-5ubuntu0.1             Xen Cloud Platform - XenAPI server

ii  xcp-xe                           1.3.2-5ubuntu0.1             Xen Cloud Platform - command-line utilities

 

Here is the vif info from the DomU (cloudstack SSVM):

# xe vif-list vm-name-label=s-1-VM 

uuid ( RO)            : d0d8489e-a3c3-54c7-32e3-8f6e07f0e60f

         vm-uuid ( RO): e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

          device ( RO): 2

    network-uuid ( RO): f3673dba-5145-641b-cab5-6f3497670540

 

 

uuid ( RO)            : cb90ccdd-858a-00a1-7ed4-c6920e5081c5

         vm-uuid ( RO): e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

          device ( RO): 1

    network-uuid ( RO): f3673dba-5145-641b-cab5-6f3497670540

 

 

uuid ( RO)            : 55ef8640-20b3-32b0-e1e6-65c3258e21d7

         vm-uuid ( RO): e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

          device ( RO): 0

    network-uuid ( RO): 22ac824e-a582-d27f-fbaa-277955854dc4

 

 

uuid ( RO)            : d14f6b5d-129a-1662-ed30-3addc00c41b3

         vm-uuid ( RO): e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

          device ( RO): 3

    network-uuid ( RO): f3673dba-5145-641b-cab5-6f3497670540

 

 

# xe network-param-list uuid=f3673dba-5145-641b-cab5-6f3497670540 

uuid ( RO)                : f3673dba-5145-641b-cab5-6f3497670540

          name-label ( RW): bond0

    name-description ( RW): 

           VIF-uuids (SRO): d14f6b5d-129a-1662-ed30-3addc00c41b3; cb90ccdd-858a-00a1-7ed4-c6920e5081c5; d0d8489e-a3c3-54c7-32e3-8f6e07f0e60f; 63bb0146-7130-53a0-a480-12bb2bd3fa9d; c27f7de3-2722-a730-815d-8569d9aef9ab

           PIF-uuids (SRO): b86c137f-544a-8ff9-98eb-c51d928536a5

                 MTU ( RW): 1500

              bridge ( RO): xapi1

        other-config (MRW): 

               blobs ( RO): 

                tags (SRW): 

 

Here is the route info from within the DomU:

# ip route show

202.56.230.5 via 192.168.1.5 dev eth1 

202.56.230.6 via 192.168.1.5 dev eth1 

192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.175 

192.168.1.0/24 dev eth2  proto kernel  scope link  src 192.168.1.245 

192.168.1.0/24 dev eth3  proto kernel  scope link  src 192.168.1.171 

169.254.0.0/16 dev eth0  proto kernel  scope link  src 169.254.0.60 

default via 192.168.1.5 dev eth2 

 

And here is the param list for the eth2 vif:

# xe vif-param-list uuid=d0d8489e-a3c3-54c7-32e3-8f6e07f0e60f 

uuid ( RO)                        : d0d8489e-a3c3-54c7-32e3-8f6e07f0e60f

                     vm-uuid ( RO): e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

               vm-name-label ( RO): s-1-VM

          allowed-operations (SRO): attach; unplug_force; unplug

          current-operations (SRO): 

                      device ( RO): 2

                         MAC ( RO): 06:a1:ce:00:00:51

           MAC-autogenerated ( RO): false

                         MTU ( RO): 1500

          currently-attached ( RO): true

          qos_algorithm_type ( RW): ratelimit

        qos_algorithm_params (MRW): kbps: 25600

    qos_supported_algorithms (SRO): 

                other-config (MRW): nicira-iface-id: 88ef7a29-9f97-4711-b884-5c6fef9082d9; nicira-vm-id: e1e4d5c9-8c36-b3b2-23e8-3a44b59878a3

                network-uuid ( RO): f3673dba-5145-641b-cab5-6f3497670540

          network-name-label ( RO): bond0

                 io_read_kbs ( RO): 3.431

                io_write_kbs ( RO): 0.000

 

Iptables on Dom0:

# iptables -L -n

Chain INPUT (policy ACCEPT)

target     prot opt source               destination         

ACCEPT     47   --  0.0.0.0/0            0.0.0.0/0           

 

Chain FORWARD (policy ACCEPT)

target     prot opt source               destination         

 

Chain OUTPUT (policy ACCEPT)

target     prot opt source               destination         

 

 

Please help.  Let me know what additional info you need.

 

thank you

Carlos

 

 

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api