Re: [Xen-devel] [xen-unstable test] 35257: regressions - FAIL

[Jim Fehlig <jfehlig@xxxxxxxx>]

Ian Campbell wrote:
> On Fri, 2015-02-27 at 10:48 +0000, Wei Liu wrote:
>   
>> On Fri, Feb 27, 2015 at 09:42:29AM +0000, Ian Campbell wrote:
>>     
>>> On Thu, 2015-02-26 at 20:14 +0000, xen.org wrote:
>>>       
>>>> flight 35257 xen-unstable real [real]
>>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/
>>>>
>>>> Regressions :-(
>>>>
>>>> Tests which did not succeed and are blocking,
>>>> including tests which could not be run:
>>>>  test-armhf-armhf-libvirt     12 guest-start.2             fail REGR. vs. 
>>>> 34629
>>>>         
>>> logs:
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/info.html
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/12.ts-guest-start.log
>>>         2015-02-23 20:21:48 Z executing ssh ... root@xxxxxxxxxxxxx virsh 
>>> domxml-from-native xen-xl /etc/xen/debian.guest.osstest.cfg > 
>>> /etc/xen/debian.guest.osstest.cfg.xml
>>>         error: failed to connect to the hypervisor
>>>         error: no valid connection
>>>         error: Failed to connect socket to '/var/run/libvirt/libvirt-sock': 
>>> Connection refused
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4-output-ps_wwwaxf_-eo_pid%2Ctty%2Cstat%2Ctime%2Cnice%2Cpsr%2Cpcpu%2Cpmem%2Cnwchan%2Cwchan%2325%2Cargs
>>> appears to show no libvirtd process.
>>>
>>> http://www.chiark.greenend.org.uk/~xensrcts/logs/35257/test-armhf-armhf-libvirt/marilith-n4---var-log-libvirt-libvirtd.log
>>> says:
>>>         2015-02-23 20:13:15.556+0000: 2133: info : libvirt version: 1.2.13
>>>         2015-02-23 20:13:15.556+0000: 2133: error : 
>>> dnsmasqCapsRefreshInternal:726 : Cannot check dnsmasq binary dnsmasq: No 
>>> such file or directory
>>>         2015-02-23 20:13:15.845+0000: 2133: error : 
>>> virFirewallValidateBackend:193 : direct firewall backend requested, but 
>>> /sbin/ebtables is not available: No such file or directory
>>>         
>>> I think these are just spurious.
>>>
>>>         2015-02-23 20:13:15.845+0000: 2133: error : virFirewallApply:936 : 
>>> out of memory
>>>         
>>>
>>>         2015-02-23 20:13:16.092+0000: 2133: error : virExec:491 : Cannot 
>>> find 'pm-is-supported' in path: No such file or directory
>>>         2015-02-23 20:13:16.092+0000: 2133: warning : virQEMUCapsInit:999 : 
>>> Failed to get host power management capabilities
>>>         
>>> As are these two.
>>>         
>>>         2015-02-23 20:13:16.400+0000: 2133: error : virFirewallApply:936 : 
>>> out of memory
>>>
>>>       
>> Last time Ian and I debugged a libvirt crashing bug, "out of memory"
>> didn't cause libvirtd to exit. It turned out it's some bug in libxl
>> event machinery that caused libvirt to exit, but the assertion message
>> was not shown anywhere.
>>
>> I think we might need to login to that host and run libvirtd in
>> foreground to determine what goes wrong.
>>     
>
> That's possible I suppose, but it would be nice to arrange not to have
> to in the future.
>
> Perhaps we should be forcing higher log levels on libvirtd when
> installing, patching /usr/local/etc/libvirt/libvirtd.conf to set
> log_level=2 (or even 1) perhaps? (Default is 3 == warnings+error, 2 is
> info, 1 is debug)
>
> Jim, what debug level would you recommend for automated test? Unless it
>   
> is super verbose I suppose 1=debug is the way to go?

I think we need DEBUG log level, although it is rather verbose.  If that
becomes a problem, we could experiment with a minimally useful
log_filters setting, e.g.

log_filters="1:daemon 1:libxl"

> Adding -v to libvirtd command line would be an easier patch, but only
> gives the effect of log_level=2 AFAICT. Perhaps that is considered
> sufficient?
>   

In my experience, if ERROR is insufficient, INFO and WARNING don't
help.  DEBUG is needed.

Regards,
Jim


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel