Re: [Xen-devel] [xen-4.8-testing test] 124100: regressions - FAIL

[Jim Fehlig <jfehlig@xxxxxxxx>]

On 06/13/2018 05:18 AM, Ian Jackson wrote:
> Jim: please read down to where I discuss
> test-amd64-amd64-libvirt-pair.  If you have any insight I'd appreciate
> it.  Let me know if you want me to preserve the logs, which will
> otherwise expire in a few weeks.
Whoa, sorry for the delay. This mail found a dumb bug in my filter for xen-devel 
mail.
> >   test-amd64-amd64-libvirt-pair 22 guest-migrate/src_host/dst_host fail pass in 
> > 123701
>  From the log:
>
> 2018-06-12 20:59:40 Z executing ssh ... root@172.16.144.61 virsh migrate --live 
> debian.guest.osstest xen+ssh://joubertin0
> error: Timed out during operation: cannot acquire state change lock
> 2018-06-12 21:00:16 Z command nonzero waitstatus 256: [..]
>
> The libvirt libxl logs seem to show libxl doing a successful
> migration.
With the long delay, I'm afraid the logs have expired. Do you still see the 
problem? All the recent runs seem to be plagued with libvirt's change to require 
GnuTLS
https://libvirt.org/git/?p=libvirt.git;a=commit;h=60d9ad6f1e42618fce10baeb0f02c35e5ebd5b24

> Looking at the logs I see this:
>
> 2018-06-12 21:00:16.784+0000: 3507: warning :
> libxlDomainObjBeginJob:151 : Cannot start job (modify) for domain
> debian.guest.osstest; current job is (modify) owned by (24947)
>
> That job number looks like it's about right for a pid, but I think it
> must be a thread because it doesn't show up in the ps output.
Likely a libvirtd worker thread doing something that requires modifying the 
state of virDomainObj.
> I did see this:
>
> Jun 12 21:00:20 joubertin0 logger: /etc/xen/scripts/vif-bridge: iptables setup 
> failed. This may affect guest networking.
>
> but that seems to be after the failure.
A wild guess, but is it possible thread 24947 is running a domain create 
operation, which includes executing vif-bridge, that is taking longer than 
expected to complete?
> I don't have an explanation.  I don't really know what this lock is.
It's a lock that serializes domain state modifications (changing virDomainObj). 
Wait time for the lock is currently hardcoded to 30sec. The thread emitting the 
warning surpassed the timeout, waiting for 24947 to finish whatever it was doing.
Regards,
Jim

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxxx
https://lists.xenproject.org/mailman/listinfo/xen-devel