|
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] Re: [Xen-devel] AMD GPU passthrough in Xen
Wednesday, September 24, 2014, 2:12:38 PM, you wrote: >> Good to know there is interest from AMD into this area :-) > I am taking a personal interest in this and would like to improve AMD support > and presence within the Xen community. Thanks for that ! > Gordan has also reported problems restarting a guest. Added Konrad to the CC as pciback maintainer. Yes, from what i see the problem seems to be that the Radeon card is not reset to a state were it will regard itself as "unposted". (on the first boot of the guest i see the driver report it hasn't posted (which is correct), it starts to past, and it works. On subsequent boots of the domU i don't see the unposted message but some failures that are probably due to the driver regarding the card as already been posted and skipping some init logic and reusing wrong values. So current xen-pciback logic doesn't seem to be able to reset the card in a "non-posted" state. I don't know if you know what kind of reset is required for radeon cards to regard itself as "not posted" ? Current xen-pciback logic uses the "__pci_reset_function_locked(dev);" functions (see pcistub.c pcistub_device_release()) to try to reset the device .. that function in turn tries some possible reset functions in a specific order and bails out at the first one reporting "succes". However it could be that this level of reset is not enough for this specific case. (in my case it always uses and succeeds at the first (pci_dev_specific_reset(dev, probe);). When looking at the code for vfio/kvm in "drivers/vfio/pci/vfio_pci.c vfio_pci_disable()", it seems to use: - Another order for disabling resetting and config save/restore - Always try a slot/bus reset on the way out .. I'm trying to experiment with that in 2 ways: 1) overrulling the logic in the domU radeon driver to do the reposting no matter in what state it thinks it is. 2) Trying to change the xen-pciback reset logic, but at present that delivers bad irq's to dom0 for completly different irq's as the passedthrough device has (which i have seen before .. and is a bit worrying) > I have been trying to reproduce the problem but have not had any luck. As a > secondary it restarts for me every time. I don't know if I inadvertently > made a change that indirectly fixed it in my code base or what the difference > might be. Perhaps in the older qemu-traditional there is already some sort of reset done ? > What Xen version are you testing with? I'm almost always living on the edge with Xen-unstable ;) (the latest and greatest, which is actually pretty stable) -- Sander > Thanks, > Kelly >> -----Original Message----- >> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx] >> Sent: Tuesday, September 23, 2014 9:45 AM >> To: Zytaruk, Kelly >> Cc: Peter Kay; xen-devel@xxxxxxxxxxxxx >> Subject: Re: [Xen-devel] AMD GPU passthrough in Xen >> >> Good to know there is interest from AMD into this area :-) >> >> I'm experimenting for a while with: >> >> - xen-unstable (and thus xl) >> - latest kernels (both dom0 and domU) >> - qemu-xen >> - Radeon HD 6570 >> - secondary passthrough >> - Debian linux (sid) with the opensource (in kernel) radeon driver >> (i also tried fglrx with succes, but it's a real PITA to build with every >> new kernel, so i ditched that) >> >> It used to work, but something broke at the moment, but that could also be >> the >> changes to the systemd cruft that Debian jessie/sid is currently undergoing >> (or >> something else since i regularly update all components). >> >> The problems are mostly with restarting the domU, it differs a bit: >> - sometimes screen goes ok, sometimes it's garbage. >> - the radeon powercontrols only seem to work on the first boot and give >> errors >> on any subsequent one. >> >> But when it works it does: >> - the powercontrol. >> - opengl and opencl benchmarks with (near) native results. >> - hardware video acceleration in xbmc for instance. >> >> So one of the main problems at present seems to be proper resetting of the >> whole device on domain shutdown/start. I did do some experiments with the >> opensource radeon driver, but didn't get conclusive results out of that yet. >> >> -- >> Sander >> >> Tuesday, September 23, 2014, 3:19:41 PM, you wrote: >> >> > Hi Peter / Sander, >> >> > Yes, I have AMD GPU passthru working as both primary and secondary >> passthru. Secondary was easy but primary is a bit tricky. >> >> > Getting on to your questions; >> >> >> Is there any specific reason you're using Xen 4.2 rather than 4.4.1? >> >> > I am working on a project that is based on Xen 4.2 (I can't say any more >> > than >> that). I have looked at some of the more recent versions just to check if >> some >> of the bugs that I have seen have been fixed but I have not studied the newer >> versions in detail. At some point in time in the future I would like will >> make the >> jump to a more recent version but I don't know the scheduling of that. >> >> >> In 4.2, using xl or xm? >> >> > xl >> >> >> qemu-traditional (with rombios) or "upstream" >> >> > qemu-traditional >> >> >> Primary or secondary passthrough? >> >> > Both but I am focusing on secondary right now. >> >> >> Presumably 64 bit versions of Windows? >> >> > 32 bit and 64 bit Win7. I have tested Win8.1 and it works but my >> > focus is currently Win7 >> >> >> I am quite willing to test various scenarios. I've a 6950, 6450 and 5450. >> >> > Awesome. My goal right now is obtaining stability on Xen 4.2. Since 4.2 >> > is >> past its feature cutoff I won't be able to submit any open source changes >> for it. >> I would like to eventually work with the community to get passthru working >> with >> a recent version of "upstream". >> >> > Thanks, >> > Kelly >> >> >> >> >> -----Original Message----- >> >> From: Sander Eikelenboom [mailto:linux@xxxxxxxxxxxxxx] >> >> Sent: Monday, September 22, 2014 8:38 AM >> >> To: Peter Kay >> >> Cc: xen-devel@xxxxxxxxxxxxx; Zytaruk, Kelly >> >> Subject: Re: [Xen-devel] AMD GPU passthrough in Xen >> >> >> >> >> >> Monday, September 22, 2014, 2:16:58 PM, you wrote: >> >> >> >> > Hi Kelly, list >> >> >> >> > I see you're having AMD GPU success with Xen 4 2 and Linux 3.4.9. >> >> > I've been >> >> less than successful getting passthrough working at all in Xen >> >> (although it's fine in KVM primary passthrough as long as the BIOS is >> >> supplied as a file). Could I confirm the following : >> >> >> >> > Is there any specific reason you're using Xen 4.2 rather than >> >> > 4.4.1? I know in >> >> some ways 4.4 suffers as it's now xl only and some of the xm >> >> functionality has not come across. >> >> >> >> > In 4.2, using xl or xm? >> >> >> >> Another interesting question/aspect would be qemu-traditional (with >> >> rombios) or "upstream" (with seabios) ? >> >> >> >> > Primary or secondary passthrough? >> >> >> >> > Presumably 64 bit versions of Windows? >> >> >> >> > My system is a bit old (Core2Quad) but as mentioned AMD passthrough >> >> > works >> >> in KVM but I've found it tricky in Xen. >> >> >> >> > I am quite willing to test various scenarios. I've a 6950, 6450 and >> >> > 5450. >> >> >> >> > Thanks >> >> >> >> > Peter >> >> >> >> >> >> _______________________________________________ Xen-devel mailing list Xen-devel@xxxxxxxxxxxxx http://lists.xen.org/xen-devel
|
 |
Lists.xenproject.org is hosted with RackSpace, monitoring our |