Re: [Xen-users] substantial shutdown delay for PV guests with PCI -passthrough

[Ian Campbell <Ian.Campbell@xxxxxxxxxx>]

On Sun, 2014-03-16 at 15:41 +0100, Atom2 wrote:
> Hi list,
> I am experiencing a substantial delay if I shutdown a PV linux guest 
> which has PCI devices passed-through:
> 
> I start the domain using
>       xl create -c domain
> 
> Then I login from the connected console and do a
>       shutdown -h now
> which results in normal shutdown messages (w/o errors) showing on the 
> console screen. The last line of messages then reads:
>       [   36.535466] reboot: System halted
> 
> So all seems normal and as expected; the strange thing, however, is that 
> the prompt in dom0 only returns after a substantial amount of time which 
> appears to be _proportional_ to the number of PCI devices passed 
> through: For every device specified in the
>       pci = [ bb:dd.f, ... ]
> entry in the guest configuration file the delay is 10 seconds. In other 
> words for 4 PCI devices passed through, it takes 40 seconds before the 
> prompt in dom0 returns (though I did not make any test with more than 4 
> devices).
> 
> If I issue the command
>       xl list
> from an other terminal connected to dom0 while the delay is on-going it 
> shows that the domain in question remains in the state 's', which 
> according to the man page indicates that "the guest OS has shut down 
> (SCHEDOP_shutdown has been called) but the domain is not dying yet". 
> This state does not change until the prompt in dom0 returns (at least 
> repeated 'xl list' commands show the same output) and then the line for 
> the domain is gone.
> 
> If no devices are passed through, the prompt in dom0 after having shut 
> down the domU appears without any noticeable delay.
> 
> All tests have been performed using the same environment while only 
> changing the relevant line in the domU configuration file.
> 
> The xen version used is 4.3.1-r5, xen-tools is also 4.3.1-r5 and the 
> kernel used for both dom0 and domUs is 3.11.7 (hardende gentoo sources).

Any chance you could try 4.3.2, or even 4.4.0?

> The system is capable of vt-d and uses a Xeon E3-1260L processor.
> 
> Do these observations ring a bell with anybody or is this even expected 
> behaviour. If this is not normal - which I would expect as I have not 
> been able to find any information relating to substantial delays during 
> shutdown - how would I go about getting to the grounds of this?

My guess would be that xl process which is managing the domain destroy
is waiting for something (perhaps pciback) to confirm shutdown for each
device and this is timing out in series, leading to the delays. You
might find something in the logs /var/log/xen pointing to something like
this.

If not then if you start the guest with "xl -vvv create -F <cfg>" then
the xl process which is monitoring the domain will stay in the
foreground and be logging to stdout (I think). If you then issue the
shutdown from another shell perhaps there will be some obvious gaps in
the logs as things shutdown which might help.

Ian.


_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users