Re: ARM/PCI passthrough: libxl_pci, sysfs and pciback questions

[Jan Beulich <jbeulich@xxxxxxxx>]

On 27.10.2020 18:45, Oleksandr Andrushchenko wrote:
> On 10/27/20 7:18 PM, Jan Beulich wrote:
>> On 27.10.2020 16:52, Oleksandr Andrushchenko wrote:
>>> On 10/27/20 2:55 PM, Roger Pau Monné wrote:
>>>> On Tue, Oct 27, 2020 at 09:59:05AM +0000, Oleksandr Andrushchenko wrote:
>>>>>     5. An alternative route for 3-4 could be to store that data in 
>>>>> XenStore, e.g.
>>>>>        MMIOs, IRQ, bind sysfs path etc. This would require more code on 
>>>>> Xen side to
>>>>>        access XenStore and won’t work if MMIOs/IRQs are passed via device 
>>>>> tree/ACPI
>>>>>        tables by the bootloaders.
>>>> As above, I think I need more context to understand what and why you
>>>> need to save such information.
>>> Well, with pciback absence we loose a "database" which holds all the 
>>> knowledge
>>>
>>> about which devices are assigned, bound etc.
>> What hasn't become clear to me (sorry if I've overlooked it) is
>> why some form of pciback is not an option on Arm.
> Yes, it is probably possible to run pciback even without running
> 
> pcifront instances in guests and only use that functionality
> 
> which is needed for the toolstack. We can even have it as is without
> 
> modifications given that pcifront won't run and that part of the pciback
> 
> related to PCI config space, MSI etc. won't simply be used, but still
> 
> present in the pciback driver. We can try that (pciback is x86
> 
> only in the kernel).
> 
>> Where it would
>> need to run in your split hardware-domain / Dom0 setup (if I got
>> that right in the first place) would be a secondary question.
> 
> This actually becomes a problem if we think about hwdom != Dom0:
> 
> Dom0/toolstack wants to read PCI bus sysfs and it also wants to access
> 
> pciback's sysfs entries. So, for Dom0's toolstack to read sysfs in this 
> scenario
> 
> we need a bridge between Dom0 and that hwdom to access both PCI
> 
> subsystem and pciback's sysfs: this could be implemented as a back-front pair
> 
> with a ring and event channel as PV drivers do. This approach of course will
> 
> require the toolstack to work in two modes: local sysfs/pciback and remote 
> ones.
> 
> In the remote access model the toolstack will need to create a connection to
> 
> the hwdom each time it runs and requires sysfs data which should be 
> acceptable.

That's the price to pay for disaggregation, I think. So yes to the
outline in general, but I'd like such an abstraction to not talk in
terms of "sysfs" or in fact anything that's OS specific on either
side. Whether it indeed needs a full new pair of front/back drivers
is a different question.

> It can also be possible to have the toolstack always use the remote model even
> 
> if it runs locally which will make the toolstack's code support a single 
> model for all
> 
> the use-cases.

That's certainly one possible way of doing the necessary abstraction,
I agree.

> (Never thought if it is possible to run both backend and frontend in the same 
> VM though).

Why would it not be? Other back/front pairs certainly can.

Jan