[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Xen-API] [Xen-devel] bug in xenstored? No notification to subscription on @introduceDomain



On 12.12.2011 17:46, Ian Campbell wrote:
Please don't top post and don't drop people/lists from the CC. I have
reinstated xen-devel and refrained from trimming the quotes as heavily
as I normally would.

Counter to my own advice I have also dropped xen-hosts@xxxxxxxxxxxxxxxx
because last time I got a bounce in Russian to the effect that the group
does not exist (according to google translate).

On Mon, 2011-12-12 at 12:10 +0000, George Shuklin wrote:
Thanks for reply.

The problem is we tried at least two different libraries - xs (+python
xen.lowlevel.xs) and our own library (pyxs), created from scratches on
pure python - both shows exactly same behavior. We loosing same time
@introduce and @release, but only for new domains. Older domains (which
starts before error appear) during shutdown/migration sends  @release
normally.

I done strace, nothig is sending by xenstored to application socket when
'new' domains appears and disappears (I'm not sure 100% due not very
good strace skills).

Application performs write/read operations to/from xenstore (and do many
subscriptions, but only after @introduce) and older subscription works fine.

PS We got other strange bug with memory leak in xenstored (happens only
with big amount of transactions, and ONLY with socket) - but this case
is still under research, so I decide not to post this (but may be it
related somehow?).
Are the two event correlated? i.e. is the oxenstored process huge when
these failures occur? Inability to allocate memory could explain some of
your symptoms although I'd expect it to be more fatal more quickly and
obviously than what you describe or to have wider impact.
Nope, memory leak occur only if transaction happens with subscription, but 'no notification' problem continues after we stops to use transaction (this cure memory leak completely, so I think this is separate issue, but I don't sure).

I still can't catch condition for lack of notifications for @introduce, sorry (I got one more this morning in test pool).

Sorry for question - how I can gather debug information for oxenstored?
What sort of debug information are you after?

There are various logging options which you could turn up to 11
in /etc/xensource/xenstored.conf but I do not have a complete list of
what they are, similarly for command line options -- perhaps someone on
xen-api@ could chime in? Otherwise looking in the source might be the
best way to find out what they are, try xenstore.ml, parse_args.ml
logging.ml would be good places to start. (if having done so you feel
motivated to write a patch to add docs/man/oxenstored.1.pod we would be
much obliged...)

Ok, thanks, I'll dig to sources to set up them all. We heavily using xenstore for dynamic memory regulation (about five operations for every domain per second).

_______________________________________________
xen-api mailing list
xen-api@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/mailman/listinfo/xen-api


 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.