Re: [Xen-devel] [Xen-users] vif-bridge errors when creating and destroying dozens of VMs simultaneously

[Roger Pau Monné <roger.pau@xxxxxxxxxx>]

On Wed, May 17, 2017 at 10:04:40AM +0100, George Dunlap wrote:
> cc'ing xen-devel & some relevant people

Please bear with me, my knowledge of iptables is 0.

> On Tue, May 16, 2017 at 4:21 PM, Antony Saba <awsaba@xxxxxxxxx> wrote:
> > Hello xen-users,
> >
> > We are seeing the following errors repeatedly while trying to create
> > domains using a script, with the end result that 2 or 3 out of about
> > 20 VMs fail to start, and there are stale entries in the iptables for
> > domains that have been destroyed.
> >
> >
> >    2017-05-10 11:45:40 UTC libxl: error:
> > libxl_exec.c:118:libxl_report_child_exitstatus:
> > /etc/xen/scripts/vif-bridge remove [18767] exited with error status 4
> >    2017-05-10 11:50:52 UTC libxl: error:
> > libxl_exec.c:118:libxl_report_child_exitstatus:
> > /etc/xen/scripts/vif-bridge offline [1554] exited with error status 4
> >
> > I've been testing the following patch of vif-common.sh over the last
> > day and it appears to resolve the issue.  iptables exits with status 4
> > when "Another app is currently holding the xtables lock."

So, an iptables command can fail randomly because there's someone else holding
an iptables internal lock?

Isn't there anyway to tell the iptables command to just block until it can get
the lock? This seems extremely racy, isn't people then forced to use something
like:

while true; do
        iptables <...>
        if [ $? == 0 ]; then
                break;
        elif [ $? != 4 ]; then
                error ...
        fi
done

When dealing with iptables?

> > Does this solution seem reasonable?

I'm not sure, this protects you from other hotplug scripts poking concurrently
at iptables, but what about the system administrator? It still seems racy to
me.

> > Thanks.
> >
> > --- /etc/xen/scripts/vif-common.sh.bak 2017-05-15 18:57:34.549288900 +0000
> > +++ /etc/xen/scripts/vif-common.sh 2017-05-15 18:58:01.361208788 +0000
> > @@ -154,12 +154,13 @@
> > # binary is not sufficient, because the user may not have the appropriate
> > # modules installed. If iptables is not working, then there's no need to do
> > # anything with it, so we can just return.
> > + claim_lock "iptables"
> > if ! iptables -L -n >&/dev/null
> > then
> > + release_lock "iptables"
> > return
> > fi
> > - claim_lock "iptables"
> > if [ "$ip" != "" ]
> > then

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel