Xen project Mailing List

On Mon, Mar 11, 2013 at 6:13 PM, Alexandre Kouznetsov <alk@xxxxxxxxxx> wrote:

Hello.

Hi. Thanks for taking the time.

El 11/03/13 18:08, Carlos Reategui escribió:

So looks like I need to setup the bonding using the xe create-bond
command just as if I were on XCP/XS. However it is not clear to me how
my /etc/network/interfaces file should be set. Does it matter? Should
I set the interfaces to manual?

I have not used XenAPI over Ubuntu (only plain Xen on Debian, and XCP), but in case of XCP it is supposed to manage the networking setup by itself.

Close enough.

eg:

auto eth0
iface eth0 inet manual

auto eth1
iface eth1 inet manual

Do I also need to set up the bridge in this file or will that be done
automatically by xcp-xapi. Eg:

auto xenbr0
iface xenbr0 inet dhcp
bridge_ports eth0

auto xenbr1
iface xenbr1 inet dhcp
bridge_ports eth1

As a reference, very unrelated to Xen-API or XCP, please note that this would not create any "bond", but a plain "bridge". The bonding magic is done by vSwitch in XCP (and probably XenAPI). Other observation is, that Debian based distribution does not need to have eth0 or eth1 interface described or set up, in order to make a valid bridge via /etc/network/interfaces file.

Understood, but do I need to define a bridge to both eth0 and 1 in the interfaces file in order to bond them. Hmm as I type that it does not sound right. If I am going to bond eth0 and 1, do I need to even bother setting up bridges for them? Seems like a bridge is a "network" in xapi which is what I'll be creating in xe.

Look at this example:
auto xenbr01
iface xenbr01 inet manual
bridge_ports eth1
bridge_stp off
bridge_maxwait 0
bridge_fd 5

This creates a bridge, set it up automatically on boot, without need to declare anything about eth1 itself. After configured, ifconfig shows eth1 in UP state, but there is no need for any "auto eth1" or "iface eth1" lines.

I guess the assumption here is that xapi picks this up when it is first setup, or do you add "xenbr01" as a network using xe yourself.

Using xe I am guessing I would then do the following:

ETH0=`xe pif-list device=eth0 --minimal`
ETH1=`xe pif-list device=eth1 --minimal`
NETID=`xe network-create name-label=bond0`
BONDID=`xe bond-create network-uuid=$NETID pif-uuids=$ETH0,$ETH1`

Note that if your host is a part of a cluster, "xe pif-list device=eth0" will return a list of PIF's instead of a single one.

The commands you listed should work, but they will bond your first two interfaces together. Your administration interface could behave funny.

By administration did you mean management? I want this bond to be my management and guest traffic interface.

To set the bond mode would I do that on NETID or BONDID above?

Consider specifying "mode=" on your "xe bond-create". In any case, the "mode" is bond's property, not network's.

good to know.

What is
the preferred bond mode on an L2 switch that only supports static LAG?

"balance-slb" or "active-backup" should work, depending on what you want. "lacp" will definitely not.

I'll try slb. It seems similar to alb.

Check this reference:
http://support.citrix.com/article/CTX124421
http://docs.vmd.citrix.com/XenServer/6.1.0/1.0/en_gb/reference.html#networking-standalone_host_config-bonds

Same with setting management to true, which UUID?

If you are talking about management interface, then it's a pif's property (the bond will have one associated).
"xe pif-list" will tell.

Ok that makes sense.

If I want to use static IP addresses would I then pif-reconfigure-ip on
the NETID?

pif-reconfigure-ip is done on PIFID, not NETID.

Ok so I do that on the bond.

What happens at reboot time will xcp take over, remember the
previous setting and ignore the interfaces file?

Here I fail you.
In XCP, the network configuration is definitely persistent across reboots, the configuration files are well hidden behind xe-toolstack.

Would it be better to
not use dhcp in the interfaces file?

Yes, much better. On any server. Except, maybe, very special cases.

I'm just going to use it for initial setup where it gets from a temporary ip pool. Then switch to static when configuring via xe.

Can I define xenbr0 and 1 in the
interfaces file as manual? I do need at least eth0 to have an IP so
that I can initially install xcp-xapi.

I would expect that manual alterations of interfaces file, once xapi is installed, would conflict with it. But never tried.
I would suggest to leave eth0 plain at the beginning, and create any bridge or other fancy configuration by using xe-toolstack means.

For my NFS network using a bond of eth2 and 3 I was thinking of keeping
them from xcp and just telling it to "xe pif-forget" on the uuids for
those. Is that ok? Im assuming as long as the host knows about that
network it should be fine.

Making this network not to be attached to the new VM's would be enough:
NEWNET=`xe network-list name-label="PUT YOURS HERE" --minimal`
xe network-param-set uuid=${NEWNET} other-config:automatic=false

If you do pif-forget, most likely xapi will be unable to manage a bond using this interface. Unless you intend to manage this interfaces completely out of the scope of xapi.

Yes. That was the plan. I was going to create an LACP bond in the interfaces file for my storage network since my storage switch does support LACP.

If this is documented somewhere let me know (I am aware of the XS docs
which do cover some of what I am asking but not in this debian
based environment).

Not the low level config, as I believe.
In XCP case (and I believe it applys to xapi as well), it is supposed to be managed via xe interface. No hand under the hood, unless you know what you are doing.
Some bit uncomfortable at the beginning.

On Sat, Mar 9, 2013 at 12:46 AM, Carlos Reategui <carlos@xxxxxxxxxxxx
<mailto:carlos@xxxxxxxxxxxx>> wrote:
In my setup xe network-list and xe pif-list are now inconsistent
with interfaces file and I think this is what is causing me some
issues.

>From XCP experience, I treat the internal configuration database as a fragile thing. So, if a hack have introduces any inconsistency, it's better to make sure that part of configuration is overwritten by xe-toolstack. Sometimes it's possible to do that without a re-install, sometimes not. For my experiments, I have set up a automated PXE installation environment, so the re-installs are not so painful.

I have 4 nics and I am creating 2 separate bonds and then
bridging them to xenbr0 and 1. One for nfs access to my primary
storage and the other for all the other vm traffic. The bonds work
great on the host but the VMs that start up are having networking
issues. This was all setup in the interfaces file and I have not
done anything network related with xe which I think is my problem
because xe network lists xenbr0-3 linked to eth0-3 instead of xenbr0
and 1 linked to bond0 and 1.

Besides what I have written above, check your interfaces naming order.

Any pointers as to the right way to do this would be greatly
appreciated.

By the book. Forget the mad skills for a while. I was unable to get anything working on XCP until I made it that way.

http://docs.vmd.citrix.com/XenServer/6.1.0/1.0/en_gb/reference.html

Greetings.

Once again thank you very much for your response. This will narrow down my experimenting and save me time. Once I get this right, I'll be adding this host to cloudstack.

Cheers,

-Carlos

--
Alexandre Kouznetsov

_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api

Re: [Xen-API] xcp-xapi bonding