Re: [Xen-devel] USB passthrough with Xen on ARM

[Roger Quadros <rogerq@xxxxxx>]

Hi,


Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki. 
Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki

On 12/09/17 18:28, Juergen Gross wrote:
> On 12/09/17 17:08, Julien Grall wrote:
>> Hello,
>>
>> On 12/09/17 12:22, Roger Quadros wrote:
>>>
>>> Texas Instruments Finland Oy, Porkkalankatu 22, 00180 Helsinki.
>>> Y-tunnus/Business ID: 0615521-4. Kotipaikka/Domicile: Helsinki
>>>
>>> On 12/09/17 13:57, Julien Grall wrote:
>>>> Hi,
>>>>
>>>> On 12/09/17 11:00, Juergen Gross wrote:
>>>>> On 12/09/17 10:13, Roger Quadros wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I'm running Xen v4.9 on DRA7 (OMAP5/ArmV7) with Linux kernel v3.14
>>>>>> (yikes!!) on dom0 and domU.
>>>>>> I'm struggling to get USB passthrough working using pvUSB.
>>>>>>
>>>>>> My domU config file contains
>>>>>>      usb = 1
>>>>>>      usbctrl = ['type=qusb,version=2,ports=4',
>>>>>> 'type=qusb,version=1, ports=4', ]
>>>>>>
>>>>>> I can see the vusb-0 and vusb-1 platform devices in /sys/devices
>>>>>>
>>>>>> And the following message on domU kernel log
>>>>>> [    1.849572] xenbus_probe_frontend: Device with no driver:
>>>>>> device/vusb/0
>>>>>> [    1.849627] xenbus_probe_frontend: Device with no driver:
>>>>>> device/vusb/1
>>>>>>
>>>>>> This means that there is no device driver for the vusb host
>>>>>> controllers.
>>>>>>
>>>>>> What is the way forward? Do I need to apply some patches to the
>>>>>> domU kernel to
>>>>>> add support for the USB frontend HCD drivers?
>>>>>
>>>>> This is one mandatory step, yes. You'll need:
>>>>>
>>>>> https://lkml.org/lkml/2015/6/23/34
>>>>> https://lkml.org/lkml/2015/6/23/36
>>>>>
>>>
>>> OK, after applying the above 2 patches to v4.12 kernel for domU I'm
>>> able to see the xen HCD driver
>>> enumerate, but it times out most likely due to the missing pvusb backend.
>>>
>>> [    0.510149] vusb vusb-0: Xen USB2.0 Virtual Host Controller
>>> [    0.510192] vusb vusb-0: new USB bus registered, assigned bus number 1
>>> [    0.510811] hub 1-0:1.0: USB hub found
>>> [    0.510865] hub 1-0:1.0: 4 ports detected
>>> [    0.812721] vusb vusb-1: Xen USB1.1 Virtual Host Controller
>>> [    0.812760] vusb vusb-1: new USB bus registered, assigned bus number 2
>>> [    0.813356] hub 2-0:1.0: USB hub found
>>> [    0.813410] hub 2-0:1.0: 4 ports detected
>>>
>>> ...
>>>
>>> [    5.888997] xenbus_probe_frontend: Waiting for devices to
>>> initialise: 25s...20s...15s...10s...5s...0s...
>>> [   35.919000]
>>> 235s...230s...225s...220s...215s...210s...205s...200s...195s...190s...185s...180s...175s...170s...165s...160s...155s...150s...145s...140s...135s...130s...125s...120s...115s...110s...105s...100s...95s...90s...85s...80s...75s...70s...65s...60s...55s...50s...45s...40s...35s...30s...25s...20s...15s...10s...5s...0s...
>>>
>>> [  270.879130]
>>> [  270.884161] xenbus_probe_frontend: Timeout connecting to device:
>>> device/vusb/0 (local state 1, remote state 1)
>>> [  270.887059] xenbus_probe_frontend: Timeout connecting to device:
>>> device/vusb/1 (local state 1, remote state 1)
>>>
>>>>> The question is whether this will be enough for you to make it work:
>>>>> the
>>>>> pvusb backend is qemu based. I'm not sure this will just work on ARM.
>>>>
>>>> Backends in QEMU should just work with ARM. Although, I haven't tried
>>>> the PVUSB one.
>>>>
>>>
>>> Newbie question now. How do I start the QEMU pvusb backend on dom0?
>>
>> If I am not mistaken, the backend should be started by
>> "/etc/init.d/xencommons start" which also start xenstore and all the
>> userspace components for Xen.
> 
> No, it should be started by libxl as a per-domain device model.
> 
>> So you should have a QEMU running in Dom0. Can you check if it is the case?
> 
> You should see whether the device model is started by doing
> 
> xl -vvv create ...
> 
> When the domain has been started you can call
> 
> xenstore-ls
> 
> to see whether there are any device model related xenstore entries.
> There should be some like:
> 
> /local/domain/0/device-model/<domid>/backends/<backend>
> 
> <backend> being "qusb" is the one you will need.
> 
>> Also, I would check that QEMU has been built with the PV USB backend. It
>> depends on libusb. I am not entirely sure if the build system will
>> require it and will therefore fault if it is not installed.
> 
> It won't. It will disable the backend without libusb being available.

This was exactly the problem. I had to rebuild xen tools with libusb-dev 
installed.
The timeout issue went away after that.

However, looks like v4.12 kernel in domU with v3.14 kernel in dom0 doesn't work 
together
as things broke when a new USB device was attached to the virtual root hub. 
Backtrace in domU
is pasted at the end.

I resolved this issue by backporting the xen-hcd kernel patches to v3.14 and 
running the v3.14
kernel in domU. Although I see this message in dom0. but it doesn't seem to 
cause any problems
with the USB device in domU in my limited tests.

[ 3495.812130] xhci-hcd xhci-hcd.1.auto: Setup ERROR: setup context command for 
slot 1.
[ 3495.932104] xhci-hcd xhci-hcd.1.auto: Setup ERROR: setup context command for 
slot 1.

One more question. Did I understand correctly that it is not possible to use 
full USB host emulation
(usbctrl type=devicemodel) with a PV guest like Linux? If this understanding is 
wrong then some pointers
on how to do that would be geat.

Thanks for the great support :)

--backtrace on v4.12 kernel on domU --

[  116.279796] usb 1-1: new high-speed USB device number 2 using vusb
[  116.279834] ------------[ cut here ]------------
[  116.279877] kernel BUG at ./include/xen/arm/page.h:84!
[  116.279905] Internal error: Oops - BUG: 0 [#1] SMP ARM
[  116.279944] Modules linked in:
[  116.279978] CPU: 0 PID: 94 Comm: kworker/0:2 Not tainted 
4.12.0-00010-ga7ecd8b #1408
[  116.280016] Hardware name: Generic DT based system
[  116.280056] Workqueue: usb_hub_wq hub_event
[  116.280084] task: da310000 task.stack: da3ae000
[  116.280112] PC is at xenhcd_do_request+0x104/0x2e4
[  116.280144] LR is at get_free_entries+0x90/0x26c
[  116.280172] pc : [<c0a7a694>]    lr : [<c07639c0>]    psr: 20000093
               sp : da3afce8  ip : 0000009c  fp : 00000001
[  116.280223] r10: db5e8000  r9 : 00000000  r8 : db5e8040
[  116.280249] r7 : da3255a0  r6 : 00000000  r5 : da5a7900  r4 : db4f8148
[  116.280278] r3 : 00000810  r2 : 00000001  r1 : 00000040  r0 : 00000000
[  116.280308] Flags: nzCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment 
none
[  116.280343] Control: 10c5387d  Table: 5a3c406a  DAC: 00000051
[  116.280374] Process kworker/0:2 (pid: 94, stack limit = 0xda3ae220)
[  116.280406] Stack: (0xda3afce8 to 0xda3b0000)
[  116.280434] fce0:                   00000001 00000000 00000036 0000009b 
c133e458 da3255a0
[  116.280470] fd00: db4f8000 da5a7900 db4f8148 db4f8168 c15df618 00000000 
a0000013 c0a7aac0
[  116.280508] fd20: c0a7a9ec da5a7900 00000000 db4f8000 01400000 da5a7908 
00000006 da41f000
[  116.280547] fd40: 00000080 c0a13550 c10560d0 da3afe14 da3afda4 c0cb10b8 
53425553 45545359
[  116.280586] fd60: 73753d4d 45440062 45434956 73752b3d 2d313a62 da3a0031 
dbbca4c0 c1402d00
[  116.280624] fd80: dbbca4c0 da3afdb8 dbbca4c0 c03a3414 c1402d00 60000013 
00000000 da3afdb8
[  116.280662] fda0: ffffb83a c03a34c4 da3afdb8 c0cb48d4 000003e8 60000013 
00000200 00000000
[  116.280700] fdc0: ffffb83a c03a348c da310000 da5a7900 da3afdf4 00000000 
da3afe24 00001388
[  116.280735] fde0: 00000006 da41f000 00000080 c0a15e28 00000001 00000000 
00000000 da3afdfc
[  116.280773] fe00: da3afdfc 80000080 da63ccc0 00000040 da63cfc0 00000100 
80000080 c0a16064
[  116.280808] fe20: da41f000 db51b800 00000000 da41f000 db51b800 da63cfc0 
00000001 00000003
[  116.280844] fe40: 00000001 00000000 db4f8000 c0a0ced8 00000100 00000000 
da63cfc0 00000040
[  116.280879] fe60: 00001388 00000001 da41f070 00001388 00000000 00000002 
00000003 00000032
[  116.280914] fe80: 00000000 00000000 000003e8 00000000 db51ba00 db591000 
db4f801c da41f000
[  116.280962] fea0: 00000001 db4f8000 db51b800 c0a0fe1c 00000000 c087a4b0 
db4f8030 00000002
[  116.280997] fec0: db591000 db5910a4 00000064 db51b600 db51ba08 db51ba00 
db51b800 db51b620
[  116.281032] fee0: db51ba00 db51ba00 db591000 db4f8000 00000000 db51b908 
00000000 00000000
[  116.281069] ff00: 00040501 c087bfe0 ffffb82d da3a3b80 db51b908 dbbcef00 
dbbcef00 dbbd3700
[  116.281107] ff20: 00000000 00000000 c1584f14 c035bc58 da6a359c da3a3b80 
dbbcef00 da3a3b80
[  116.281144] ff40: dbbcef18 00000001 dbbcef00 da3a3b98 dbbcef00 00000008 
c1402d00 c035bfa4
[  116.281182] ff60: db09fee0 da2f7a00 00000000 da2f7a00 00000000 da2f7fc0 
da2f7a1c da3a3b80
[  116.281217] ff80: db09fee0 c035bf78 00000000 c0361458 da2f7fc0 c0361338 
00000000 00000000
[  116.281262] ffa0: 00000000 00000000 00000000 c0308938 00000000 00000000 
00000000 00000000
[  116.281302] ffc0: 00000000 00000000 00000000 00000000 00000000 00000000 
00000000 00000000
[  116.281337] ffe0: 00000000 00000000 00000000 00000000 00000013 00000000 
5bfde861 5bfdec61
[  116.281381] [<c0a7a694>] (xenhcd_do_request) from [<c0a7aac0>] 
(xenhcd_urb_enqueue+0xd4/0x118)
[  116.281424] [<c0a7aac0>] (xenhcd_urb_enqueue) from [<c0a13550>] 
(usb_hcd_submit_urb+0xac/0x868)
[  116.281467] [<c0a13550>] (usb_hcd_submit_urb) from [<c0a15e28>] 
(usb_start_wait_urb+0x44/0xbc)
[  116.281508] [<c0a15e28>] (usb_start_wait_urb) from [<c0a16064>] 
(usb_control_msg+0x9c/0xd0)
[  116.281546] [<c0a16064>] (usb_control_msg) from [<c0a0ced8>] 
(hub_port_init+0x480/0xa28)
[  116.281585] [<c0a0ced8>] (hub_port_init) from [<c0a0fe1c>] 
(hub_event+0x638/0xff0)
[  116.281641] [<c0a0fe1c>] (hub_event) from [<c035bc58>] 
(process_one_work+0x144/0x42c)
[  116.281686] [<c035bc58>] (process_one_work) from [<c035bfa4>] 
(worker_thread+0x2c/0x4f0)
[  116.281737] [<c035bfa4>] (worker_thread) from [<c0361458>] 
(kthread+0x120/0x158)
[  116.281789] [<c0361458>] (kthread) from [<c0308938>] 
(ret_from_fork+0x14/0x3c)
[  116.281831] Code: d6ff2072 da000004 e3510000 0a000052 (e7f001f2) 
[  116.281866] ---[ end trace b84b26d49eeaf778 ]---


> 
>> Juergen, do you have other idea why it would timeout?
> 
> qemu not running or without qusb support is the most probable one.
> 
> 
> Juergen
> 

-- 
cheers,
-roger


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel