[Xen-users] 2 recently updated Centos6 Xen 4.6.3-5 server crashes

["Brandon Shoemaker" <brandon.shoemaker@xxxxxxxxx>]

Hello list,

I recently upgraded our Centos6 Xen servers to 4.6.3-5 about three weeks ago 
and strangely this week we have had 2 servers crash with similar issues.  
Nothing is able to be logged in /var/log/messages because the problem appears 
to be disk related as the problem develops.  I took screenshots of console for 
each server.

https://postimg.org/image/3xmuun3gv/
https://postimg.org/image/mw6ukvd93/

It appears there are I/O errors, then the journal aborts on the root logical 
volume, and then there is EXT4-fs errors, and then swap errors reported.

Our monitoring manages to capture this information remotely before the crash.  
It was the same for both servers.

CRIT - filesystem has switched to read-only and is probably corrupted CRIT, 
missing: rw, exceeding: ro

Centos 6.8 3.18.44-20.el6.x86_64
xen version 4.6.3-5.el6
48gb RAM
Samsung 850 pro 1 TB SSD (yes I know the consumer class of this drive)
X10SRi-F motherboard 1.0b and 2.0a bios (two different servers running two 
different BIOS versions)

One server was about half full and the other server near full as for VPS disk 
allocation but had plenty of CPU and RAM resources available.  Neither server 
has heavy I/O characteristics.

The servers have been in use without issue for months prior to this.

I find it unlikely that both servers would have a SSD disk fail in the same 
exact manner within a few days of each other so I'm suspicious that this is a 
software bug.

I'm wondering if this could have been caused by the recent Xen and or operating 
system updates since it is coincidental that I am now having this issue after 
recently updating when before the servers were stable.

Has anyone else had any similar crashes recently like this?


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
https://lists.xen.org/xen-users