Re: [Xen-devel] Issue with MSI in a HVM domU with several passed through PCI devices

[Stefano Stabellini <stefano.stabellini@xxxxxxxxxxxxx>]

On Mon, 25 Jun 2012, Jan Beulich wrote:
> >>> On 24.06.12 at 04:21, Rolu <rolu@xxxxxxxx> wrote:
> > On Wed, Jun 20, 2012 at 6:03 PM, Jan Beulich <JBeulich@xxxxxxxx> wrote:
> >> At the same time, adding logging to the guest kernel would
> >> be nice, to see what value it actually writes (in a current
> >> kernel this would be in __write_msi_msg()).
> >>
> > 
> > Turns out that msg->data here is also 0x4300, so it seems the guest
> > kernel is producing these values. I caused it to make a stack trace
> > and this pointed back to xen_hvm_setup_msi_irqs. This function uses
> > the macro XEN_PIRQ_MSI_DATA, which evaluates to 0x4300. It checks the
> > current data field and if it isn't equal to the macro it uses
> > xen_msi_compose_msg to make a new message, but that function just sets
> > the data field of the message to XEN_PIRQ_MSI_DATA - so, 0x4300. This
> > then gets passed to __write_msi_msg and that's that. There are no
> > other writes through __write_msi_msg (except for the same thing for
> > other devices).
> > 
> > The macro XEN_PIRQ_MSI_DATA contains a part (3 << 8) which ends up
> > decoded as the delivery mode, so it seems the kernel is intentionally
> > setting it to 3.
> 
> So that can never have worked properly afaict. Stefano, the
> code as it is currently - using literal (3 << 8) - is clearly bogus.
> Your original commit at least had a comment saying that the
> reserved delivery mode encoding is intentional here, but that
> comment got lost with the later introduction of XEN_PIRQ_MSI_DATA.
> In any case - the cooperation with qemu apparently doesn't
> work, as the reserved encoding should never make it through
> to the hypervisor. Could you explain what the intention here
> was?
> 
> And regardless of anything, can the literal numbers please be
> replaced by proper manifest constants - the "8" here already
> has MSI_DATA_DELIVERY_MODE_SHIFT, and giving the 3 a
> proper symbolic would permit locating where this is being (or
> really, as it doesn't appear to work supposed to be) consumed
> in qemu, provided it uses the same definition (i.e. that one
> should go into one of the public headers).

The (3 << 8) is unimportant. The delivery mode chosen is "reserved"
because notifications are not supposed to be delivered as MSI anymore.

This is what should happen:

1) Linux configures the device with a 0 vector number and the pirq number
in the address field;

2) QEMU notices a vector number of 0 and reads the pirq number from the
address field, passing it to xc_domain_update_msi_irq;

3) Xen assignes the given pirq to the physical MSI;

4) The guest issues a EVTCHNOP_bind_pirq hypercall;

5) Xen sets the pirq as "IRQ_PT";

6) When Xen tries to inject the MSI into the guest, hvm_domain_use_pirq
returns true so Xen calls send_guest_pirq instead.


Obviously 6) is not happening. hvm_domain_use_pirq is:

is_hvm_domain(d) && pirq && pirq->arch.hvm.emuirq != IRQ_UNBOUND

My guess is that emuirq is IRQ_UNBOUND when it should be IRQ_PT (see
above).
So maybe the guest is not issuing a EVTCHNOP_bind_pirq hypercall
(__startup_pirq doesn't get called), or Xen is erroring out in
map_domain_emuirq_pirq.

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel