[Xen-users] problem with static routes

[Greg Woods <woods@xxxxxxxx>]

The basic problem is that when I reboot a node in my cluster, it comes
back up without its static routes.

I posted this first to the linux-ha list, but I have since determined
that the problem happens when Xen is stopped or started. The original
post appears below. Since then, I have determined that if I add code to
my bridge-wrapper script (which basically calls the network-bridge
script once for each interface) that manually adds the static routes
back in, the problem goes away on startup, but still occurs when I put a
node into standby (which takes down all the shared resources including
Xen). 

I realize that I am running an old version of Xen, but I am mostly
wondering if this behavior is intentional or if there is a less kludgy
workaround (like a config parameter that can be set that I have missed).

Am I the only one in the world who has to use static routes?
Unfortunately I am stuck with them because we have a /16 address space
that is partially inside and partially outside our security perimeter,
which means some subnets are reached through the external interface and
some through the internal one; these are defined with static routes.

TIA,
--Greg

==============================================================================
OS: CentOS 5.5
heartbeat: heartbeat-3.0.3-2.3.el5 (latest from clusterlabs)
pacemaker: pacemaker-1.0.9.1-1.15.el5 (latest from clusterlabs)

If it matters, this cluster is primarily used to run Xen virtual
machines (xen-3.0.3-105.el5_5.5 kernel-2.6.18-194.11.1.el5xen latest
from CentOS)

I have been looking off and on for the source of this problem for quite
a while without finding what is causing it. The basic problem is that
when I reboot a node in my cluster, it comes back up without its static
routes. Adding them back in manually works; they stay until the next
reboot. These are defined in /etc/sysconfig/static-routes and are added
by the network service at boot time. I have been able to pretty much
rule out the boot process itself as the source of the problem. I added a
"netstat -r -n > /tmp/static-routes" command to the rc.local file which
is the very last thing run at boot time and the routes are there. I have
also tried putting nodes into standby (crm node standby) and back
online, and the routes stay there through that. But once I log in after
a reboot, the static routes are gone and I have to manually re-add them.

I can probably work around this using a hideous kludge like having the
rc.local file run a background job that sleeps for a couple of minutes,
then adds the routes, but that doesn't really fix the issue and isn't
guaranteed to work reliably (obviously high reliability is important or
I wouldn't be using HA in the first place).

Has anyone ever seen this before or have any clue where I can look to
troubleshoot this?

Thanks in advance,
--Greg


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users