[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Xen-users] PCIe Device Driver


  • To: "xen-users@xxxxxxxxxxxxx" <xen-users@xxxxxxxxxxxxx>
  • From: Kenneth Wong <kenwong@xxxxxxxxxxx>
  • Date: Wed, 23 May 2012 16:40:08 -0700
  • Accept-language: en-US
  • Acceptlanguage: en-US
  • Delivery-date: Fri, 25 May 2012 15:00:03 +0000
  • List-id: Xen user discussion <xen-users.lists.xen.org>
  • Thread-index: Ac05PWOLdUDHBp99R1+dOk4Gtppm8w==
  • Thread-topic: PCIe Device Driver

Dear all,

 

I have a PCIe device driver that I have been using on various Linux distributions and Kernel versions (2.6.x – 3.x.y) successfully all along. 

 

I recently set up a Xen environment with Linux Mint 12 and Xen Hypervisor 4.1.  When I boot to Linux Mint, my driver still load (via insmod manually) successfully at Dom0 without any issue.  I can do reads and write to the hardware device.  But once booted to Xen, the driver failed to complete the driver load (via insmod manually) at Dom 0 and the console just hangs.

 

From my debug messages, it appears it hangs because the driver doesn’t receive any interrupt after a command is sent to the hardware device by writing a parameter to the mapped register.  Once that register is written, the device is expected to DMA the command from the buffer allocated by the driver.

 

The things that I can only think of that might have caused the problem are 1) IRQ mapping issue, or 2) DMA mapping issue, which I am not sure.

 

 

What the driver does:

 

Set up a command buffer:

Buf_t *buf = kmalloc(BUF_SIZE*sizeof(buf_t), GFP_KERNEL);

unsigned long buf_addr = __pa(buf);

unsigned int buf_addr_low = (unsigned int)buf_addr;

 

Tell device about the buffer:

iowrite32(buf_addr_low, dev->pci_reg_map + BUF_ADR__LOW);

 

Set up IRQ:

    if (pci_find_capability(dev, PCI_CAP_ID_MSI) &&

        (!pci_enable_msi(dev)))

    {

        if (request_irq(dev->irq, func_msi_interrupt, IRQF_SHARED, DRIVER_NAME, my_dev))

        {

            return  -ENODEV;

        }

        my_dev->intr_mode = INTERRUPT_MSI;

    }

 

Ask device to fetch command from buffer (Expect interrupt after this after device fetched the command from buf.  But interrupt did not happen.):

iowrite32(buf_offset, dev->pci_reg_map + FETCH_CMD_REG);

 

 

From dmesg, it looks like IRQ initialization is complete.

[  241.743769] My_driver initialization

[  241.743787] xen: registering gsi 16 triggering 0 polarity 1

[  241.743793] xen_map_pirq_gsi: returning irq 16 for gsi 16

[  241.743795] xen: --> pirq=16 -> irq=16 (gsi=16)

[  241.743801] Already setup the GSI :16

[  241.743805] my-driver 0000:02:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16

[  241.743815] my-driver 0000:02:00.0: setting latency timer to 64

 

/proc/interrupts:

            CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7

……

……

339:          0          0          0          0          0          0          0          0  xen-pirq-msi       my-driver

……

……

 

Any idea what might cause the problem?

 

Is there anything we have to be enable/disable, use different functions, or do differently in drivers written for Xen Dom0 environment regarding the following?

1)       Allocating a DMA buffer in driver to allow the device to DMA stuffs.

2)       Requesting MSI irq.

 

Please advise!

 

Thanks a lot in advance!!

 

Kenneth

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users

 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.