Re: [Xen-users] 101th domU fails to start with "SETVCPUCONTEXT failed"

[Lennert Van Alboom <lennert.vanalboom@xxxxxxxxxx>]

On Sun, Jan 10, 2010 at 12:30:12AM +0100, storm66@xxxxxxxxxxxxxxxx wrote:
> Le samedi 09 janvier 2010 à 21:57 +0100, Lennert Van Alboom a écrit :
> > Hello there,
> >
> >
> > We (a small hosting community) are running a steadily growing number of Xen 
> > domUs on a quad dualcore Xeon server with 64GB ram. We've got 100 running 
> > domUs at the moment. Trying to create a new one results in this error:
> > Error: (1, 'Internal error', 'launch_vm: SETVCPUCONTEXT failed (rc=-1)\n')
> >
> > If I shut down another domain, I can create one again, but the limit
> > seems to be 100 domUs. 
> > Each domU has one network interface and one or two disk devices (raid-1
> > md device consisting of two iscsi luns).
> >
> > Version of Xen is rather old: 3.2.1-amd64 (Debian). 
> >
> > Can anyone give me hint as to what causes this behaviour? The host
> > machine should be able to cope with the amount of domUs easily. I've got
> > a feeling I'm bumping into an obscure sourcecode parameter somewhere.
> >
> >
> > Thanks in advance,
> >
> >
> > Lennert
> Hello,
>
> In my thought Xen is heavily using loop devices, I had some problems
> some years ago with a system running XEN 3.0 with a kernel where the
> default loop max number was too low. 
> I didn't get that problem with newer versions.
>
> Regards
>
> JPP
Thanks for your reply. We're not using loopbacks though - only physical
devices (raid1 md). 

I've been tracking this problem through the xen source, and found this:
The launch_vm code is located in
http://lxr.mstier.de/Xen/source/xen_3.2.1/tools/libxc/xc_dom_boot.c?v=3.2.1#049
; the function that is causing the return code of -1 is do_domctl.

I'm unsure as to where do_domctl is really located - I've found two
locations:
http://lxr.mstier.de/Xen/source/xen_3.2.1/tools/libxc/xc_private.h?v=3.2.1#099
and
http://lxr.mstier.de/Xen/source/xen_3.2.1/xen/common/domctl.c?v=3.2.1#179
. Neither give me much of a clue. 

In case of the first definition, the returncode is given by 
ret = do_xen_hypercall(xc_handle, &hypercall)
which leads me further to 
ioctl(xc_handle, cmd, data);
and after that I'm stumped. 

In the second case, the -1 comes *either* from 
IS_PRIV(current->domain)
failing, causing an EPERM error. This sounds unlikely. The other place
where a -1 could be given is 
ret = xsm_setvcpucontext(d);
which translates back to
xsm_call(setvcpucontext(d));
and there I'm stuck again. I'm totally not capable of deciphering
xsm_ops and its meanings.


Does anyone with some insight in the innards of Xen have a clue on what
might be causing our issues?


Kind regards,


Lennert



_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users