Re: [Xen-devel] PCI Passthrough ARM Design : Draft1

[Julien Grall <julien.grall@xxxxxxxxxx>]

Hi Ian,

On 17/06/15 11:08, Ian Campbell wrote:
> On Tue, 2015-06-16 at 18:16 +0100, Stefano Stabellini wrote:
>> I wrote this before reading the rest of the thread with your alternative
>> suggestion.  Both approaches can work, maybe it is true that having the
>> guest requesting mappings is better. And we should certainly do the same
>> thing for PVH and ARM guests.
>>
>> If we have the guest requesting the mapping, obviously the hypercall
>> implementation should check whether mapping the memory region has been
>> previously allowed by the toolstack, that should still call
>> xc_domain_iomem_permission.
> 
> Whoever makes the initial decision/setup we still need to decide what to
> do about reads and writes to the device's BAR registers. Who deals with
> these and how, and if writes are allowed and if so how is this then
> reflected in the guest p2m.
> 
> I would very much prefer ARM and x86 PVH to use the same approach to
> this.
> 
> For x86 HVM I believe QEMU takes care of this, by emulating the
> reads/writes to CFG space and making hypercalls (domctls?) to update the
> p2m. There is no QEMU to do this in the ARM or x86 PVH cases.
> 
> For x86 PV I believe pciback deals with the register reads/writes but it
> doesn't need to do anything wrt the p2m because that is a guest internal
> construct. This obviously doesn't apply to ARM or x86 PVH either since
> the p2m is managed by Xen in both those cases.

By default, the OS doesn't setup the BARs, it expects the firmware to do
it. If I'm not mistaken the PV guest is using the same memory layout as
the host, so the BAR is mapped 1:1 in the guest.  The interesting part
is bar_* in drivers/xen/xenpci-back/config_space_header.c

> So it seems that neither of the existing solutions are a good match for
> either ARM or x86 PVH.
> 
> _If_ it were permissible to implement all BAR registers as r/o then this
> might simplify things, however I suspect this is not something we can
> get away with in the general case (i.e. a driver for a given PCI device
> _knows_ that BAR<N> is writeable...).

The guest has to write in the BAR in order to get the size of the BAR [1].

> 
> So I think we do need to allow guest writes to the BAR registers. I
> think it would be a bit strange (or even open potentially subtle
> (security?) issues) to require the guest to write the BAR register and
> to update the p2m to match as well.

I don't see any possible security issue to let the guest map the BAR
itself. If the guest doesn't have the right to some physical region it
won't be able to map it (see iomem permission).

Actually, we already do similar things for interrupt.

> 
> So I think updates to the BAR need to be reflected in the p2m too by
> whichever entity is emulating those reads/writes.
> 
> QEMU (or another daemon) is not really an option IMHO.

Well, by default the OS doesn't setup the BAR and expect the firmware to
initialize the BAR. In our case, there is no firmware and Xen will act
as it. It would be OK, to do it in the toolstack at boot time.

> Which leaves us with either Xen doing trap and emulate or pciback
> dealing with this via the pciif.h cfg space access stuff. I've a slight
> preference for the latter since I think both ARM and x86 PVH are
> planning to use pcifront/pciback already.

pcifront/pciback is already emulate the access to the config space for us...

It would be mad to use Xen for that as we would have to reinvent our own
config space access way (ARM doesn't have ioport, hence cf8 is not present).

> 
> Which leaves us with a requirement for the backend to be able to update
> the p2m, which in turn leads to a need for an interface available to the
> dom0 kernel (which the existing domctl is not).
> 
> There will also need to be a mechanism to expose a suitable MMIO hole to
> pcifront. On x86 this would be via the machine memory map, I think, but
> on ARM we may need something else, perhaps a xenstore key associated
> with the pci bus entry in xenstore?
> 
> That's for guests, for dom0 Xen also need to be aware of changes to the
> BAR registers of the real physical devices since it needs to know the
> real hardware values in order to point guest p2m mappings at them.

I don't understand why dom0 Xen needs to be aware of changes in physical
BAR.

PCIback is already able to deal with the value of physical BAR, you have
nothing else to do (see drivers/xen/xenpci-back/config_space_header.c).

Regards,

[1] http://wiki.osdev.org/PCI#Base_Address_Registers

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
http://lists.xen.org/xen-devel