Re: [PATCH v1 2/6] xen/arm: Reserve resources for virtio-pci

[Julien Grall <julien@xxxxxxx>]

Hi Edgar,

On 24/09/2024 18:11, Edgar E. Iglesias wrote:
> On Tue, Sep 24, 2024 at 05:35:20PM +0100, Julien Grall wrote:
> > Hi Edgar,
> >
> > On 24/09/2024 17:23, Edgar E. Iglesias wrote:
> > > From: "Edgar E. Iglesias" <edgar.iglesias@xxxxxxx>
> > >
> > > Reserve memory ranges and interrupt lines for an externally
> > > emulated PCI controller (e.g by QEMU) dedicated to hosting
> > > Virtio devices and potentially other emulated devices.
> > >
> > > Signed-off-by: Edgar E. Iglesias <edgar.iglesias@xxxxxxx>
> > > ---
> > >    xen/include/public/arch-arm.h | 17 +++++++++++++++++
> > >    1 file changed, 17 insertions(+)
> > >
> > > diff --git a/xen/include/public/arch-arm.h b/xen/include/public/arch-arm.h
> > > index e19f0251a6..654b827715 100644
> > > --- a/xen/include/public/arch-arm.h
> > > +++ b/xen/include/public/arch-arm.h
> > > @@ -494,6 +494,20 @@ typedef uint64_t xen_callback_t;
> > >    #define GUEST_RAM1_BASE   xen_mk_ullong(0x0200000000) /* 952GB of RAM @ 8GB 
> > > */
> > >    #define GUEST_RAM1_SIZE   xen_mk_ullong(0xee00000000)
> > > +/* Virtio PCI - Ordered by decreasing size to keep things aligned */
> > > +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_TYPE  xen_mk_ullong(0x43000000)
> > > +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_BASE  xen_mk_ullong(0x0f000000000)
> > > +#define GUEST_VIRTIO_PCI_PREFETCH_MEM_SIZE  xen_mk_ullong(0x100000000)
> > > +
> > > +#define GUEST_VIRTIO_PCI_ECAM_BASE      (GUEST_VIRTIO_PCI_PREFETCH_MEM_BASE + \
> > > +                                         GUEST_VIRTIO_PCI_PREFETCH_MEM_SIZE)
> > > +#define GUEST_VIRTIO_PCI_ECAM_SIZE      xen_mk_ullong(0x10000000)
> > > +
> > > +#define GUEST_VIRTIO_PCI_MEM_TYPE         xen_mk_ullong(0x02000000)
> > > +#define GUEST_VIRTIO_PCI_MEM_BASE         (GUEST_VIRTIO_PCI_ECAM_BASE + \
> > > +                                           GUEST_VIRTIO_PCI_ECAM_SIZE)
> > > +#define GUEST_VIRTIO_PCI_MEM_SIZE         xen_mk_ullong(0x00002000000)
> > Why is this specific to virtio PCI? However, I am not entirely convinced we
> > should have a second PCI hostbridge exposed to the guest for a few reasons:
> >    1. This require to reserve yet another range in the address space (could
> > be solved with a more dynamic layout)
> >    2. From your instructions, the guest needs to explicitly do a PCI rescan.
Another big advantage I forgot to mention is disaggregation. In a world 
where the hostbridge is handled in Xen, you could have a PCI device 
emulated by dom0 while the other is emulated by a stub domain.
> > So rather than having a second hostbridge, have you considered to extend the
> > existing hostbridge (implemented in Xen) to support a mix of physical PCI
> > device and virtual one?
> Thanks Julien,
>
> It's briefly come up in a couple of discussions but I haven't looked
> carefully at it. It is a good idea and it's probably worth prototyping
> to see what the gaps are in hypercall interfaces, QEMU support etc.
I also vaguely recall to discuss it on xen-devel. But I couldn't find 
the discussion... :(.
I think all the hypercalls should be there but will require some 
plumbing in Xen on Arm. QEMU should be able to request Xen to forward 
configuration access for a given PCI device (see XEN_DMOP_IO_RANGE_PCI). 
They will then be forwarded to QEMU using IOREQ_TYPE_PCI_CONFIG.
We also have an hypercall to be able to inject interrupts from QEMU (see 
XEN_DMOP_set_irq_level).
Cheers,

--
Julien Grall