[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-devel] Fwd: Re: MSI regression with xen hvm domUs using virtio devices

I'm forwarding this to xen-devel too.
I have also tried with latest xen-unstable (commit feee1ace547cf6247a358d082dd64fa762be2488) and also with latest qemu upstream unstable (commit 8cfc114a2f293c40077d1bdb7500b29db359ca22) but the problem persist.
Il 03/01/2014 22:14, Konrad Rzeszutek Wilk ha scritto:
> On Thu, Dec 19, 2013 at 01:03:35PM +0100, Fabio Fantoni wrote:
>> Hi, sorry for bothering you.
>> Virtio devices work on xen hvm domUs on windows and on linux with
>> old kernel (for example Squeeze with kernel 2.6.32), while on
>> Precise, Wheezy, Saucy and Sid with kernel >=3.2 they work only with
>> pci=nomsi on the kernel boot line.
> OK.
>> I tried debian experimental kernel (3.12.3) with this patch you made:
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0e4ccb1505a9e29c50170742ce26ac4655baab2d
>> but the problem persists.
> Right, that fixes a different issue.
>> If you know something else that I can try to solve this problem I'm
>> all ears.
>> Otherwise where could I report this problem?
> So, what is it that does not work?

Thanks for your reply.
All virtio devices with xen hvm domUs with kernel > 2.6.32 don't work 
without adding pci=nomsi on linux boot kernel line. (the regression 
could be between 2.6.32 and 3.2, I not tested kernels between 2.6.32 and 
I tested with spice vdagent (that use virtio-serial) and virtio-net.
I've used spice vdagent since end of 2011, mainly on window domUs. It 
works correctly and also with xen pv driver loaded and is on xen upstream:
Virtio-net can be setted simply on xl cfg, for example: 
vif=['model=virtio-net-pci,bridge=xenbr0'], even if is not officially 
supported on xen upstream for now and was working since wei liu "virtio 
on xen" project:
virtio on xen originally (on 2011) required pci=nomsi in order to works 
but after this Wei Liu qemu patch (see below) it works out of the box:
About virtio-net there is a regression in qemu 1.6, I have narrowed down 
with bisect (one commit between 4 Jul 2013 and 22 Jul 2013). But this is 
little out of topic.

What I'm interested in is mainly to solve this regression/problem with 
kernel > 2.6.32 with virtio on xen in order to have vdagent working out 
of the box in all cases. (also with newer kernel without pci=nomsi)

>   Do you see errors on the console?
> Does /proc/interrupts show any number of MSI interrupts going up?
> Is there something obvious in /var/log/xen/qemu-* ?

Only these:
xc: error: linux_gnttab_set_max_grants: ioctl SET_MAX_GRANTS failed (22 
= Invalid argument): Internal error
xen be: qdisk-768: xc_gnttab_set_max_grants failed: Invalid argument
but I think that are not related to this problem, are showed also on 
domUs without virtio devices.

> What does the lspci look like in your earlier kernsl (The ones that
> worked?) OR your /proc/interrupts?

/proc/interrupts seems different from old kernel working without 
pci=nomsi and new kernel with both pci=nomsi and not.

On attachments some logs about 3 tests:
- squeeze (debian 6 with kernel 2.6.32) with vdagent and virtio-net, 
working (*-squeeze.txt), cmdline: pci=nomsi xen_emul_unplug=never (for 
- precise (ubuntu 12.04 LTS) with vdagent and virtio-net, working 
(*-precise-nomsi.txt), cmdline: pci=nomsi xen_emul_unplug=never (for 
- precise (ubuntu 12.04 LTS) with vdagent, not working 
(*-precise-onlyvdagent.txt), cmdline: (blank)

The third test have also xl -vvv create and xl dmesg logs only vdagent 
to have all working also with xen pv enabled.

When virtio are not working I not saw particular error, only that 
vdagent is not working because does not create 

The only thing I have noticed is the interrupt difference on virtio 
devices beetwen squeeze test (PCI-MSI-edge), precise test working 
(xen-pirq-ioapic-level) and precise not working (xen-pirq-msi-x).

If you need more details and/or tests tell me and I'll post them.

Thanks for any reply.

Attachment: proc-interrupts-squeeze.txt
Description: Text document

Attachment: lspci-precise-nomsi.txt
Description: Text document

Attachment: proc-interrupts-precise-nomsi.txt
Description: Text document

Attachment: xl-create-precise-onlyvdagent.txt
Description: Text document

Attachment: xl-dmesg-precise-onlyvdagent.txt
Description: Text document

Attachment: lspci-precise-onlyvdagent.txt
Description: Text document

Attachment: proc-interrupts-precise-onlyvdagent.txt
Description: Text document

Attachment: lsipci-squeeze.txt
Description: Text document

Xen-devel mailing list



Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.