Re: [Xen-devel] how to generate a smaller core with xm dump-core

[Don Slutz <dslutz@xxxxxxxxxxx>]

On 01/15/15 05:20, Ian Campbell wrote:
> On Thu, 2015-01-15 at 11:31 +0800, Zhenzhong Duan wrote:
> > Hi Maintainers,
> >
> >
> > We are facing issue collecting  coredump using the xm dump mechanism
> > in Dom0.
> >
> > We face couple of such issues daily, where the VMs panic s and the SA
> > team is supposed to collect the core.
> >
> > The actual problem is that where the VMs are having huge RAM like 32+
> > GB RAMs dumping the core using xm dump generated core of almost the
> > same size.
> >
> > And also when the RAM size is huge the time taken to dump is also
> > huge.
> >
> > Transferring this core dump of this huge size to another system for
> > analysis is really a pain. I am looking for some solution where we can
> > generate smaller core using xm dump or any other tool.
> >
> > Something similar to makedumpfile which eliminates the zero and
> > unnecessary pages, which results in core of smaller size.
> >
> >   
> >
> > This has become a hot issue and we terribly need this feature.
> >
> > Can you come across with anything or are you aware of any feature
> > which could cater our needâ.?
> I'm not aware of any existing features or efforts in this area.
>
> If things like zero pages can be omitted from a core file without making
> it non-standard then I think we'd be happy to see such patches to the
> core libxc functions which produce the core files.
>
> That wouldn't help much with a heavily loaded VM which is making good
> use of all its ram though.
>
> I've not looked but it's not unlikely that the dumping code itself is
> not very optimal, so it might be worth having a look to see if more
> batching etc might help speed things up at the root. Ages ago there were
> patches floating around which improved the performance of the privcmd
> mmap ioctl (from Mats Peterson), which might help here too (assuming
> core dumping uses those interfaces as I expect). AFAIK those patches
> were abandoned when Mats moved onto other things, resurrecting them
> would probably help here as well as with migration and other
> foreign-memory-man heavy workloads.
>
> Other random thoughts:
>        * Automatically compress the core file, doesn't help with time to
>          produce the dump (quite the opposite) but helps with copying it
>          off box (but could also compress manually before moving or do it
>          as a post processing step)
>        * Have an option to produce some kind of "minicore", e.g. just
>          register state and a few pages of the referenced stacks, perhaps
>          a few pages of RAM either side of any register which happens to
>          be a valid looking pointer.
>                * Care would be needed to retain an acceptable probability
>                  that the resulting core will actually be useful.
>        * Consider including kernel RAM only (which might still be huge
>          though if it has a 1:1 mapping of all RAM).
>
> I don't think any of those would be suitable for enabling by default but
> they might be useful to have in the arsenal.
>
> HTH,
>
> Ian.
If these are Linux guests then the patch:

http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02351.html

Can be used to enable crash to access the crashed guest and collect
some basic info. I would also include the output of xen-hvmctx and/or 
xenctx.
-- This is the quick way I would do Ian's minicore .

Note: dump-core currently does not include xen-hvmctx output (nor does 
it include
xenctx -a output).

Using the results from this may allow you to not need a copy of every 
dump (
not to imply that 2 similar minicore's would insure that a copy of each core
was not needed).

I also think that makedumpfile can process a xen dump-core file and make it
smaller.

    -Don Slutz

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxx
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel