Re: [Xen-API] xcp-xapi bonding

[Alexandre Kouznetsov <alk@xxxxxxxxxx>]

Hello.

El 11/03/13 18:08, Carlos Reategui escribió:
> So looks like I need to setup the bonding using the xe create-bond
> command just as if I were on XCP/XS.  However it is not clear to me how
> my /etc/network/interfaces file should be set.  Does it matter?  Should
> I set the interfaces to manual?
I have not used XenAPI over Ubuntu (only plain Xen on Debian, and XCP), 
but in case of XCP it is supposed to manage the networking setup by itself.
> eg:
>
> auto eth0
> iface eth0 inet manual
>
> auto eth1
> iface eth1 inet manual
>
> Do I also need to set up the bridge in this file or will that be done
> automatically by xcp-xapi. Eg:
>
> auto xenbr0
> iface xenbr0 inet dhcp
> bridge_ports eth0
>
> auto xenbr1
> iface xenbr1 inet dhcp
> bridge_ports eth1
As a reference, very unrelated to Xen-API or XCP, please note that this 
would not create any "bond", but a plain "bridge". The bonding magic is 
done by vSwitch in XCP (and probably XenAPI). Other observation is, that 
Debian based distribution does not need to have eth0 or eth1 interface 
described or set up, in order to make a valid bridge via 
/etc/network/interfaces file.
Look at this example:
auto xenbr01
iface xenbr01 inet manual
  bridge_ports eth1
  bridge_stp off
  bridge_maxwait 0
  bridge_fd 5

This creates a bridge, set it up automatically on boot, without need to 
declare anything about eth1 itself. After configured, ifconfig shows 
eth1 in UP state, but there is no need for any "auto eth1" or "iface 
eth1" lines.
> Using xe I am guessing I would then do the following:
>
> ETH0=`xe pif-list device=eth0 --minimal`
> ETH1=`xe pif-list device=eth1 --minimal`
> NETID=`xe network-create name-label=bond0`
> BONDID=`xe bond-create network-uuid=$NETID pif-uuids=$ETH0,$ETH1`
Note that if your host is a part of a cluster, "xe pif-list device=eth0" 
will return a list of PIF's instead of a single one.
The commands you listed should work, but they will bond your first two 
interfaces together. Your administration interface could behave funny.
> To set the bond mode would I do that on NETID or BONDID above?
Consider specifying "mode=" on your "xe bond-create". In any case, the 
"mode" is bond's property, not network's.
>  What is
> the preferred bond mode on an L2 switch that only supports static LAG?
"balance-slb" or "active-backup" should work, depending on what you 
want. "lacp" will definitely not.
Check this reference:
http://support.citrix.com/article/CTX124421
http://docs.vmd.citrix.com/XenServer/6.1.0/1.0/en_gb/reference.html#networking-standalone_host_config-bonds

>   Same with setting management to true, which UUID?
If you are talking about management interface, then it's a pif's 
property (the bond will have one associated).
"xe pif-list" will tell.

> If I want to use static IP addresses would I then pif-reconfigure-ip on
> the NETID?
pif-reconfigure-ip is done on PIFID, not NETID.

>  What happens at reboot time will xcp take over, remember the
> previous setting and ignore the interfaces file?
Here I fail you.
In XCP, the network configuration is definitely persistent across 
reboots, the configuration files are well hidden behind xe-toolstack.
>  Would it be better to
> not use dhcp in the interfaces file?
Yes, much better. On any server. Except, maybe, very special cases.

>  Can I define  xenbr0 and 1 in the
> interfaces file as manual?  I do need at least eth0 to have an IP so
> that I can initially install xcp-xapi.
I would expect that manual alterations of interfaces file, once xapi is 
installed, would conflict with it. But never tried.
I would suggest to leave eth0 plain at the beginning, and create any 
bridge or other fancy configuration by using xe-toolstack means.
> For my NFS network using a bond of eth2 and 3 I was thinking of keeping
> them from xcp and just telling it to "xe pif-forget" on the uuids for
> those.  Is that ok?  Im assuming as long as the host knows about that
> network it should be fine.
Making this network not to be attached to the new VM's would be enough:
NEWNET=`xe network-list name-label="PUT YOURS HERE" --minimal`
xe network-param-set uuid=${NEWNET} other-config:automatic=false

If you do pif-forget, most likely xapi will be unable to manage a bond 
using this interface. Unless you intend to manage this interfaces 
completely out of the scope of xapi.
> If this is documented somewhere let me know (I am aware of the XS docs
> which do cover some of what I am asking but not in this debian
> based environment).
Not the low level config, as I believe.
In XCP case (and I believe it applys to xapi as well), it is supposed to 
be managed via xe interface. No hand under the hood, unless you know 
what you are doing.
Some bit uncomfortable at the beginning.


> On Sat, Mar 9, 2013 at 12:46 AM, Carlos Reategui <carlos@xxxxxxxxxxxx
> <mailto:carlos@xxxxxxxxxxxx>> wrote:
>     In my setup xe network-list and xe pif-list are now inconsistent
>     with interfaces file and I think this is what is causing me some
>     issues.
From XCP experience, I treat the internal configuration database as a 
fragile thing. So, if a hack have introduces any inconsistency, it's 
better to make sure that part of configuration is overwritten by 
xe-toolstack. Sometimes it's possible to do that without a re-install, 
sometimes not. For my experiments, I have set up a automated PXE 
installation environment, so the re-installs are not so painful.
>  I have 4 nics and I am creating 2 separate bonds and then
>     bridging them to xenbr0 and 1. One for nfs access to my primary
>     storage and the other for all the other vm traffic.  The bonds work
>     great on the host but the VMs that start up are having networking
>     issues.  This was all setup in the interfaces file and I have not
>     done anything network related with xe which I think is my problem
>     because xe network lists xenbr0-3 linked to eth0-3 instead of xenbr0
>     and 1 linked to bond0 and 1.
Besides what I have written above, check your interfaces naming order.

>     Any pointers as to the right way to do this would be greatly
>     appreciated.
By the book. Forget the mad skills for a while. I was unable to get 
anything working on XCP until I made it that way.
http://docs.vmd.citrix.com/XenServer/6.1.0/1.0/en_gb/reference.html

Greetings.

--
Alexandre Kouznetsov


_______________________________________________
Xen-api mailing list
Xen-api@xxxxxxxxxxxxx
http://lists.xen.org/cgi-bin/mailman/listinfo/xen-api