[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] Issues with PCI-Passtrough (VT-d) in HVM with Xen 4.6
On 3 Jun 2016, at 13:52, Jan Beulich wrote: On 02.06.16 at 21:59, <syso@xxxxxxxxxxxxx> wrote:Temporary Fix ------------- After checking the source[1] of the PCI configuration space handling in xen-pciback, I found out that changing line 258 to read if (handled && !err) { instead of: if (!handled && !err) { solves the issue and I can successfully write to the interface.This can't possibly a solution (see also the comment right there). But without you telling us what register gets written with what value, and what's being read back, I don't think there's much we can do or suggest. In the end, by changing the conditional you force writes that _did_ get emulated to _nevertheless_ also get carried out to raw hardware. If that helps for your specific case, then it's quite possible that something in the emulation paths invoked from the code above that conditional is wrong. Many thanks Jan for your comment!!Regarding the flow of data: there is a user level "toolkit" and the actual uio driver (uio_netx). 1. The uio_netx driver "only" configures the device and turns the interrupts off. This appears to be working, although I find it strange, that when it is "write request 4 bytes" conf_space.c seems to do it twice, when it is "write request 2 bytes", it is only once (see below) - but I am not sure if I overlooked something. 2. the user level driver part (toolkit) first reads out the PCI config, send a reset sequence to the device, and then restores (write back the previously save PCI config) to the driver. After that, it should upload the firmware blob to the PCI card, which does not happen, because it thinks, the reset was not successful. But I need to capture this again and compare with the settings of the reset sequence in the source of the driver (will send later). So far I got the following- does that make sense to you? Dom0: lspci -vvv -s 06:00.0 06:00.0 Unassigned class [ff00]: Hilscher GmbH CIFX 50E-DP(M/S)Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx- Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 64 Interrupt: pin A routed to IRQ 11 Region 0: Memory at f7a00000 (32-bit, non-prefetchable) [size=64K] modprobe xen-pciback verbose_request=1 xl pci-assignable-add 06:00.0 xl pci-attach debian-guest 06:00.0,permissive=1Driver gets automatically loaded in DomU and the driver tries to disable the interrupts https://github.com/torvalds/linux/blob/master/drivers/uio/uio_netx.c#L99 on Dom0: tail -f /var/log/kern.log ---- snip ----pciback 0000:06:00.0: enabling permissive mode configuration space accesses! [139971.850691] pciback 0000:06:00.0: permissive mode is potentially unsafe! [139971.904249] xen_pciback: vpci: 0000:06:00.0: assign to virtual slot 0 [139971.906548] pciback 0000:06:00.0: registering for 17766 [139971.914313] xen-pciback: 0000:06:00.0: read 4 bytes at 0x0 [139971.914327] xen-pciback: 0000:06:00.0: read 4 bytes at 0x0 = 15cf [139971.914341] xen-pciback: 0000:06:00.0: read 1 bytes at 0xe [139971.914350] xen-pciback: 0000:06:00.0: read 1 bytes at 0xe = 0 [139971.914356] xen-pciback: 0000:06:00.0: read 2 bytes at 0x6 [139971.914364] xen-pciback: 0000:06:00.0: read 2 bytes at 0x6 = 200 [139971.914378] xen-pciback: 0000:06:00.0: read 4 bytes at 0x8[139971.914386] xen-pciback: 0000:06:00.0: read 4 bytes at 0x8 = ff000000 [139971.914402] xen-pciback: 0000:06:00.0: read 2 bytes at 0x6 [139971.914409] xen-pciback: 0000:06:00.0: read 2 bytes at 0x6 = 200 [139971.914423] xen-pciback: 0000:06:00.0: read 1 bytes at 0x3d [139971.914437] xen-pciback: 0000:06:00.0: read 1 bytes at 0x3d = 1 [139971.914448] xen-pciback: 0000:06:00.0: read 1 bytes at 0x3c [139971.914456] xen-pciback: 0000:06:00.0: read 1 bytes at 0x3c = 10 [139971.914466] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 [139971.914480] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 = 103[139971.914490] xen-pciback: 0000:06:00.0: write request 2 bytes at 0x4 = 100 [139971.914492] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 [139971.914505] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 = 103 ---- snap ----which is from here https://github.com/torvalds/linux/blob/master/drivers/xen/xen-pciback/conf_space.c#L208 One observation that struck me:- if the write request is a word (as opposed to a double word), the write always seems to succeed! [139971.914490] xen-pciback: 0000:06:00.0: write request 2 bytes at 0x4 = 100 <---- input value [139971.914492] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4[139971.914505] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 = 103 <---- before write [139971.914527] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4[139971.914540] xen-pciback: 0000:06:00.0: read 2 bytes at 0x4 = 100 <---- after write - if the write request is a double word, the write _sometimes_ seems to succeed! or is this just some method the overwrite all registers with "ffffffff" first and then set the actual value? [914572] xbk: 06:00.0: write request 4 bytes at 0x10 = ffffffff [914574] xbk: 06:00.0: read 4 bytes at 0x10 [914582] xbk: 06:00.0: read 4 bytes at 0x10 = f7a00000 [914591] xbk: 06:00.0: read 4 bytes at 0x10 [914599] xbk: 06:00.0: read 4 bytes at 0x10 = 10000 <---- fail -- [914636] xbk: 06:00.0: write request 4 bytes at 0x10 = f7a00000 [914638] xbk: 06:00.0: read 4 bytes at 0x10 [914646] xbk: 06:00.0: read 4 bytes at 0x10 = 10000 [914660] xbk: 06:00.0: read 4 bytes at 0x10[914667] xbk: 06:00.0: read 4 bytes at 0x10 = f7a00000 <---- success -- [919484] xbk: 06:00.0: write request 4 bytes at 0x14 = ffffffff [919486] xbk: 06:00.0: read 4 bytes at 0x14 [919493] xbk: 06:00.0: read 4 bytes at 0x14 = 0 [919502] xbk: 06:00.0: read 4 bytes at 0x14 [919508] xbk: 06:00.0: read 4 bytes at 0x14 = 1 <---- fail -- [919542] xbk: 06:00.0: write request 4 bytes at 0x14 = 0 [919544] xbk: 06:00.0: read 4 bytes at 0x14 [919551] xbk: 06:00.0: read 4 bytes at 0x14 = 1 [919564] xbk: 06:00.0: read 4 bytes at 0x14 [919574] xbk: 06:00.0: read 4 bytes at 0x14 = 0 <---- success -- [922060] xbk: 06:00.0: write request 4 bytes at 0x18 = ffffffff [922062] xbk: 06:00.0: read 4 bytes at 0x18 [922069] xbk: 06:00.0: read 4 bytes at 0x18 = 0 [922082] xbk: 06:00.0: read 4 bytes at 0x18 [922089] xbk: 06:00.0: read 4 bytes at 0x18 = 1 <---- fail -- [922136] xbk: 06:00.0: write request 4 bytes at 0x18 = 0 [922138] xbk: 06:00.0: read 4 bytes at 0x18 [922146] xbk: 06:00.0: read 4 bytes at 0x18 = 1 [922158] xbk: 06:00.0: read 4 bytes at 0x18 [922166] xbk: 06:00.0: read 4 bytes at 0x18 = 0 <---- successSo far, this seems to be fine, actually. The problem starts, once we run the diagnostic tool for the PCI card (works fine of course on Dom0), which initiates the reset on the card (again: read PCI config, send reset sequence, write back config to PCI card, check if some "flag" is valid (which is not), and then goes on to upload the firmware to the device). Many thanks!!! Jürgen _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
![]() |
Lists.xenproject.org is hosted with RackSpace, monitoring our |