Dear all,
I have a PCIe device driver that I have been using on
various Linux distributions and Kernel versions (2.6.x – 3.x.y) successfully
all along.
I recently set up a Xen environment with Linux Mint 12 and
Xen Hypervisor 4.1. When I boot to Linux Mint, my driver still load (via
insmod manually) successfully at Dom0 without any issue. I can do reads
and write to the hardware device. But once booted to Xen, the driver
failed to complete the driver load (via insmod manually) at Dom 0 and the
console just hangs.
From my debug messages, it appears it hangs because the
driver doesn’t receive any interrupt after a command is sent to the
hardware device by writing a parameter to the mapped register. Once that
register is written, the device is expected to DMA the command from the buffer
allocated by the driver.
The things that I can only think of that might have caused
the problem are 1) IRQ mapping issue, or 2) DMA mapping issue, which I am not
sure.
What the driver does:
Set up a command buffer:
Buf_t *buf = kmalloc(BUF_SIZE*sizeof(buf_t),
GFP_KERNEL);
unsigned long buf_addr = __pa(buf);
unsigned int buf_addr_low =
(unsigned int)buf_addr;
Tell device about the buffer:
iowrite32(buf_addr_low,
dev->pci_reg_map + BUF_ADR__LOW);
Set up IRQ:
if
(pci_find_capability(dev, PCI_CAP_ID_MSI) &&
(!pci_enable_msi(dev)))
{
if (request_irq(dev->irq, func_msi_interrupt, IRQF_SHARED, DRIVER_NAME, my_dev))
{
return -ENODEV;
}
my_dev->intr_mode = INTERRUPT_MSI;
}
Ask device to fetch command from buffer (Expect interrupt after
this after device fetched the command from buf. But interrupt did not
happen.):
iowrite32(buf_offset,
dev->pci_reg_map + FETCH_CMD_REG);
From dmesg, it looks like IRQ initialization is complete.
[
241.743769] My_driver initialization
[
241.743787] xen: registering gsi 16 triggering 0 polarity 1
[
241.743793] xen_map_pirq_gsi: returning irq 16 for gsi 16
[
241.743795] xen: --> pirq=16 -> irq=16 (gsi=16)
[
241.743801] Already setup the GSI :16
[
241.743805] my-driver 0000:02:00.0: PCI INT A -> GSI 16 (level, low) ->
IRQ 16
[
241.743815] my-driver 0000:02:00.0: setting latency timer to 64
/proc/interrupts:
CPU0
CPU1
CPU2
CPU3 CPU4
CPU5
CPU6 CPU7
……
……
339:
0
0
0
0
0
0
0 0
xen-pirq-msi my-driver
……
……
Any idea what might cause the problem?
Is there anything we have to be enable/disable, use
different functions, or do differently in drivers written for Xen Dom0 environment
regarding the following?
1) Allocating a
DMA buffer in driver to allow the device to DMA stuffs.
2) Requesting MSI
irq.
Please advise!
Thanks a lot in advance!!
Kenneth
