[Xen-devel] stubdom fs-backend issue

[Keith Coleman <list.keith@xxxxxxxxxxx>]

I initially encountered this issue testing live migration with stubdoms. The first 19 live migrations work but the domU and domU-dm freeze on the 20th attempt. The same behavior is observed when doing save/restore cycles.

 

The issue seems to be related to fs-backend. After fs-backend has launched 20 stubdoms it is no longer able to restore a domU.

 

Restarting fs-backend after 19 uses avoids this issue but orphans any running stubdoms, causing them to lock up.

 

I have tried various flavors of linux and windows in domU. On dom0 I have tried xen 3.3.0, 3.3.1, 3.3.2-rc1-pre on the official 2.6.18.8 and the latest lenny dom0 kernel. I tried 32bit and amd64. The issue exists across the board.

 

To help debug the problem I am attaching several logs. Here are the steps I took:

strace fs-backend 1>fs-backend-strace-1 2>fs-backend-strace-2

xm create win3-stub

xm save win3 win3.save; sleep 2; xm restore win3.save   (did this 19 times)

xm save win3 win3.save

xm restore win3.save

At this point the domU and domU-dm are not functioning:

xm list
Name                                        ID   Mem VCPUs      State   Time(s)
Domain-0                                     0   512     2     r-----    254.6
win3                                        41   512     1     ------      0.0
win3-dm                                     42    32     1     -b----      0.0

 

destroyed win3 and killed fs-backend

 

I have attached the last several qemu logs that show the domUs working and the last one that didn't work, xend.log (starting when I booted the dom0 machine), domain-builder-ng.log and the output of stracing fs-backend:

 

qemu-dm-win3-dm.log
qemu-dm-win3-dm.log.1
qemu-dm-win3-dm.log.2
qemu-dm-win3-dm.log.3
qemu-dm-win3.log
qemu-dm-win3.log.1
qemu-dm-win3.log.2
qemu-dm-win3.log.3
xend.log

domain-builder-ng.log

fs-backend-strace-1 (~90MB uncompressed)
fs-backend-strace-2

I hope someone can take a few minutes to reproduce this issue. I mentioned it on the users list but got no responses and haven't been able to find anyone successfully using stubdoms that has time to test. When stubdoms are working they work so well that I don't want to give up.

 

Thanks,

Keith Coleman

Attachment: fs-backend-error-00.tgz.sha1sum
Description: Text document

Attachment: fs-backend-error-00.tgz
Description: GNU Zip compressed data

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel