Re: [Xen-users] regession: domU locks up with kernel 4.1.10

[Sven KÃhler <sven.koehler@xxxxxxxxx>]

Dear Ian,

Am 12.10.2015 um 11:09 schrieb Ian Campbell:
> On Sun, 2015-10-11 at 22:51 +0200, Sven KÃhler wrote:
>> Hi,
>>
>> while 4.1.8 runs solid for months, 4.1.10 locks up minutes or even
>> seconds after domU has booted. The log is full of messages like
>>
>> INFO: rcu_sched detected stalls on CPUs/tasks: { 0}
>>
>> and xl top shows the corresponding domU stuck at above 100-104% CPU
>> usage. Also, the domU becomes unresponsive as far as network I/O is
>> concerned.
>>
>> I'm using the kernel sources from kernel.org without any patches. I use
>> the same .config for 4.1.8 and 4.1.10.
>>
>> git diff --stat didn't show any suspecious xen related changes, so I'm
>> wondering what's going on. Is anybody else seeing this?
> 
> FWIW the Xen Project osstest automated test has been running on 4.1.10 and
> isn't seeing anything like this. It's seeing an unrelated and apparently
> machine specific failure to migrate an HVM guest, but the other stuff is wo
> rking ok.

It might be related to my kernel config.

> Is it the dom0 or the domU which is locking up? Which are you changing?

I'm changing just the domU kernel, nothing else.

> The rcu_sched stall message is a generic symptom of lots of potential
> issues (it just means some CPU got stuck), the surrounding logs would
> likely contain more information. It is best to post the whole thing.

There is nothing before the first stalls are detected.

> It's possible that some generic or arch/x86 change has interacted badly
> with the Xen support, or that this is e.g. a driver issue.

Could be.

> 4.1.8 to 4.1.10 is only 190 commits, which is only a handful of steps with
> git bisect, so it would be well worth trying that.

I upgraded a domU which is a productive system, kind of. I'd like to
avoid any downtime or risking data loss at this point. I have no idea
yet, whether the issue can be reproduced one a test system. For a test
system, I have no real hardware, just a VirtualBox machine to test on.

I will try to dig deeper (and git bisect if I can reproduce the issue)
once I have proper Internet connection again.


Regards,
  Sven

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users