Re: [Xen-users] Network and SATA Instability on Xen 4.6/4.8

[Kevin Stange <kevin@xxxxxxxxxxxxx>]

On 12/08/2017 05:57 PM, Sarah Newman wrote:
> On 12/08/2017 03:28 PM, Kevin Stange wrote:
>> On 12/08/2017 04:50 PM, Sarah Newman wrote:
>>> On 12/08/2017 01:17 PM, Kevin Stange wrote:
>>>>
>>>> I don't know if this is a bug in Xen or something else at play, but I
>>>> could really use some help figuring out what's going on, why msi=off
>>>> seems to fix it, and if there are any better ways to resolve this.
>>>>
>>>> Thanks.
>>>>
>>>
>>> Do you mind sharing your exact lspci output and xen and linux command lines?
>>
>> Not at all.
>>
>> # xl info | grep command
>> xen_commandline        : placeholder dom0_mem=1535M cpuinfo
>> com1=115200,8n1 console=com1,tty loglvl=all guest_loglvl=all
>> dom0_max_vcpus=2 msi=off
> 
> 1535M is kind of an unusual amount of ram, don't you think? It shouldn't 
> matter but have you tried something a bit more round, maybe at least on a
> 4MiB boundary?

I'll try that.  This specific number is due to some math that the
management software install script does at setup.  Sometimes the
reported RAM is not quite 100%.  In this case it should have come out to
1536 MB (1.5 GB), but the reported total memory is 73718 MB (10 MB short
of 72 GB) and basically just truncated the result to an int.

> You might want to check if there were any changes to the typical memory map 
> in between 4.4 and 4.6 and review at your BIOS settings. I vaguely recall
> BIOS settings related to limiting memory ranges and PCI. Maybe the old 
> version of Xen was accidentally enforcing that.

I don't really know where I'd need to look for the aforementioned
changes or how to measure them against my BIOS options.  Can you point
me at any more detailed resources for this?

>> # cat /proc/cmdline
>> placeholder root=UUID=58788945-d0e9-4a76-87ce-4308e86b8fd6 ro nomodeset
>> crashkernel=auto rd.md.uuid=88fbb495:49586ca8:0d13f6b4:86500d08
>> rd.md.uuid=c00da35e:8bf4e8f6:b90591c9:dc98206d
>> rd.md.uuid=c996bde2:6b217770:e8822aa5:42722e67 rhgb quiet pcie_aspm=off
>> console=hvc0 earlyprintk=xen nomodeset
> 
> We run with edd=off but no modifiers to pci.

pcie_aspm=off was a shot in the dark, but didn't do anything to help.
We had some X8SIE-F motherboards which had a problem under the CentOS 6
kernel with NIC behavior and this option fixed that, so it's just one of
the things I throw at the wall when I'm having PCI problems, especially
if NIC related.  I don't think it's doing anything on this server.  I
wouldn't mind if ASPM was just not a thing on servers, though.

pci=nomsi was something I thought I might try next as well, but msi=off
on the Xen line seems to mitigate the problem so far.

-- 
Kevin Stange
Chief Technology Officer
Steadfast | Managed Infrastructure, Datacenter and Cloud Services
800 S Wells, Suite 190 | Chicago, IL 60607
312.602.2689 X203 | Fax: 312.602.2688
kevin@xxxxxxxxxxxxx | www.steadfast.net

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-users