[Xen-users] Xen 4.3 Passthrough Problems & Documentation

[Casey DeLorme <cdelorme@xxxxxxxxx>]

Hello Xen Users,

I have attached the latest copy of my own documentation in PDF format as well as some error files related to the latest tests I ran. ÂI have documented the steps I took to reach my current situation. ÂWhile I have a "working" install, it is not optimal and there are several problems and questions I would like to address:

- If Dom0 is a privileged DomU, should we be enabling the DomU kernel flags in addition to the Dom0 kernel flags?

- What purposes does xen-watchdog server, is it only for xend, and do we still need it for xl?

- What benefits can be had by limiting Dom0 vcpu's in grub?

- Upstream qemu fails to load virtual machines with VGA passthrough and a large amount of memory (3600MB+ in my case breaks the DomU).

- Does anyone know exactly what Windows device ejection does to the hardware, or how we can do the same from Linux (such as Dom0)?

I would appreciate any suggestions or answers that you can provide to these questions.

---

**A note on GPLPV:**

Using the latest GPLPV, and so far it works excellent. ÂTo be honest I don't notice a different with regards to disk IO, solid state is already fast, but the Windows Experience index jumps from a 6.6 to a 7.9.

What is really astounding is network traffic. ÂI run a DomU Samba File Server and without GPLPV I manage 50~Mbps for file transfers. ÂWith GLPV that jumps to an average over 200~Mbps.

---

**A note on passthrough and upstream qemu:**

I tried followed David Techer's latest blog post, but I don't use the xm toolstack or qemu-traditional. ÂI only wanted the memory patch for passthrough with upstream, since I am still limited to 3GB.

I followed these steps to build:

  git clone git://xenbits.xen.org/xen.git

  cd xen

  git checkout -b stable-4.3 origin/stable-4.3

  cd tools

  make -j9 && make clean

  cd qemu-xen-dir

  wget "http://marc.info/?l=qemu-devel&m=136177475215360&q=raw" -O - | patch -p1

  cd ../..

  make -j9 world && make -j9 debball

  cd dist

  dpkg -i *.deb

The patch applied successfully, and xen built and installed without a problem. ÂHowever, when I attempted to boot my Windows DomU (no changes to the config yet just to see if it would boot), it failed. ÂI have attached the output, as well as the BSoD screen Windows displayed over VNC.

Without the patch process (lines `cd tools` to `cd ../..`), I can boot my Windows DomU but I cannot supply large amounts of memory.

---

Testing sysfs reset:

Modern linux kernel sysfs comes with reset files that can be used to reset (some) PCI devices manually:

- [Kernel Docs (https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci)](https://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci)

I decided to give this a try to see if it would allow me to reset the adapter from within Linux, where I could then tie a script to automate the reset process when a DomU is rebooted.

The planned scenario:

- Windows boots and initializes the graphics card

- I shut down windows and the card remains initialized

- I reset the graphics card state by:

  - Unbinding from pciback

  - issuing a reset

  - rebinding it

- Booting windows should initialize a fresh card

I decided to try the following set of commands and reboot my Windows DomU between each to see if performance was still a concern (duplicate commands for audio device omitted):

First I tried simply unbinding and rebinding to pciback:

  echo 0000:03:00.0 > /sys/bus/pci/devices/0000\:03\:00.0/driver/unbind

  echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/new_slot

  echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/bind

This did nothing so I tried unbinding, resetting, and rebinding:

  echo 0000:03:00.0 > /sys/bus/pci/devices/0000\:03\:00.0/driver/unbind

  echo 1 > /sys/bus/pci/devices/0000\:03\:00.0/reset

  echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/new_slot

  echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/bind

This did not work, and also threw an error when I attempred to reset the device. ÂFrom various tests I found that the reset would fail if the card was not bound to a driver or enable contained a 0. Âleaving it bound or echoing a 1 into the enable file before issuing a reset still did not resolve the degradation.

Some possible conclusions I drew from these failures:

- I am to believe that Windows ejection is probably working because it is using AMD drivers.

- The reset in Linux fails when it has no drivers so the reset probably triggers a driver operation

- The driver operation probably fails because it is not tied to an AMD driver

Another option I have not yet tested would be loading the radon driver to bind and unbind it before adding it back to pciback, which may cause the proper reset chain to occur. ÂI didn't see it in the drivers list though and wouldn't know where to begin loading it without causing problems.

If anyone knows how to cause a D0>D3>D0 power change to a device through sysfs let me know because I would like to try that next.

Thank you for your time,

~Casey

Attachment: Tentative Xen 4.3 System Setup Documentation.pdf
Description: Adobe PDF document

Attachment: bsod.png
Description: PNG image

Attachment: qemu-dm-windows.log
Description: Binary data

Attachment: verbose_create.log
Description: Binary data

Attachment: xl-windows.log
Description: Binary data

_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxx
http://lists.xen.org/xen-users