[Xen-devel] Please help: domU becomes unresponsive

["Jerry Amundson" <jamundso@xxxxxxxxx>]

Hi all, sorry to intrude on xen-devel, but I think I need direction
from the expertise here. I've admin'd Xen servers of various flavors
for a couple years, but never seen this before. After a period ranging
from several hours to several days, my  primary database and
development DomU completely locks up. Net disconnects, but CPU(sec)
continues to tick in xentop. No errors, and nothing logged. All dom's
are CentOS, so I'm pasting below what I've already posted to
centos-devel and centos-virt.

On Mon, Jul 14, 2008 at 3:49 PM, Jerry Amundson <jamundso@xxxxxxxxx> wrote:
> Two Dell 6950 (now called R905, 4 Dual-Core AMD Opteron 8200 series)
> heartbeat/drbd nodes running the stock CentOS 5.2 Dom0. The domU's are
> the only resources in heartbeat.
> Dom1 is a perfectly running, updated, CentOS 5.2 Apache/MySQL/Samba
> Dom2 is a CentOS 4.6 software development and database server

So crash tells me that Dom2 gets to this point:
 SYSTEM MAP: System.map-2.6.9-67.0.20.ELxenU
DEBUG KERNEL: /usr/lib/debug/lib/modules/2.6.9-67.0.20.ELxenU/vmlinux
(2.6.9-67.0.20.ELxenU)
   DUMPFILE: /public/IntSys/tmp/m1.dmp
       CPUS: 6
       DATE: Mon Jul 14 11:53:59 2008
     UPTIME: 6 days, 11:39:33
LOAD AVERAGE: 548.07, 542.95, 434.99
      TASKS: 2721
   NODENAME: monolith
    RELEASE: 2.6.9-67.0.20.ELxenU
    VERSION: #1 SMP Thu Jun 26 08:36:44 EDT 2008
    MACHINE: x86_64  (2194 Mhz)
     MEMORY: 10 GB
      PANIC: ""
        PID: 0
    COMMAND: "swapper"
       TASK: ffffffff80322b40  (1 of 6)  [THREAD_INFO: ffffffff80426000]
        CPU: 0
      STATE: TASK_RUNNING
    WARNING: panic task not found

crash> bt
PID: 0      TASK: ffffffff80322b40  CPU: 0   COMMAND: "swapper"
 #0 [ffffffff80427ec0] schedule at ffffffff80294d9a
 #1 [ffffffff80427f98] cpu_idle at ffffffff8010b85d
crash> kmem -i
             PAGES        TOTAL      PERCENTAGE
 TOTAL MEM  2621696        10 GB         ----
     FREE     8884      34.7 MB    0% of TOTAL MEM
     USED  2612812        10 GB   99% of TOTAL MEM
   SHARED        0            0    0% of TOTAL MEM
  BUFFERS    59585     232.8 MB    2% of TOTAL MEM
   CACHED  1325825       5.1 GB   50% of TOTAL MEM
     SLAB   358565       1.4 GB   13% of TOTAL MEM

TOTAL HIGH        0            0    0% of TOTAL MEM
 FREE HIGH        0            0    0% of TOTAL HIGH
 TOTAL LOW  2621696        10 GB  100% of TOTAL MEM
 FREE LOW     8884      34.7 MB    0% of TOTAL LOW

kmem: swap_info[0].swap_map at ffffff00001ea000 is unaccessible

So I see where the DomU is, but how did it get there? Can I find out
from crash, or do I need something "real-time" within the DomU? Of
course, searching has given me nothing to go on, hence this post, but
I'll continue...

Tia,
jerry

-- 
"Your life is trite and jaded, boring and confiscated." - Twisted Sister

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel