Re: Virtio in Xen on Arm (based on IOREQ concept)

[Alex Bennée <alex.bennee@xxxxxxxxxx>]

Julien Grall <julien@xxxxxxx> writes:

> (+ Andree for the vGIC).
>
> Hi Stefano,
>
> On 20/07/2020 21:38, Stefano Stabellini wrote:
>> On Fri, 17 Jul 2020, Oleksandr wrote:
>>>>> *A few word about solution:*
>>>>> As it was mentioned at [1], in order to implement virtio-mmio Xen on Arm
>>>> Any plans for virtio-pci? Arm seems to be moving to the PCI bus, and
>>>> it would be very interesting from a x86 PoV, as I don't think
>>>> virtio-mmio is something that you can easily use on x86 (or even use
>>>> at all).
>>>
>>> Being honest I didn't consider virtio-pci so far. Julien's PoC (we are based
>>> on) provides support for the virtio-mmio transport
>>>
>>> which is enough to start working around VirtIO and is not as complex as
>>> virtio-pci. But it doesn't mean there is no way for virtio-pci in Xen.
>>>
>>> I think, this could be added in next steps. But the nearest target is
>>> virtio-mmio approach (of course if the community agrees on that).
>
>> Aside from complexity and easy-of-development, are there any other
>> architectural reasons for using virtio-mmio?
>
<snip>
>> 
>> For instance, what's your take on notifications with virtio-mmio? How
>> are they modelled today?
>
> The backend will notify the frontend using an SPI. The other way around 
> (frontend -> backend) is based on an MMIO write.
>
> We have an interface to allow the backend to control whether the 
> interrupt level (i.e. low, high). However, the "old" vGIC doesn't handle 
> properly level interrupts. So we would end up to treat level interrupts 
> as edge.
>
> Technically, the problem is already existing with HW interrupts, but the 
> HW should fire it again if the interrupt line is still asserted. Another 
> issue is the interrupt may fire even if the interrupt line was 
> deasserted (IIRC this caused some interesting problem with the Arch timer).
>
> I am a bit concerned that the issue will be more proeminent for virtual 
> interrupts. I know that we have some gross hack in the vpl011 to handle 
> a level interrupts. So maybe it is time to switch to the new vGIC?
>
>> Are they good enough or do we need MSIs?
>
> I am not sure whether virtio-mmio supports MSIs. However for virtio-pci, 
> MSIs is going to be useful to improve performance. This may mean to 
> expose an ITS, so we would need to add support for guest.

virtio-mmio doesn't support MSI's at the moment although there have been
proposals to update the spec to allow them. At the moment the cost of
reading the ISR value and then writing an ack in vm_interrupt:

        /* Read and acknowledge interrupts */
        status = readl(vm_dev->base + VIRTIO_MMIO_INTERRUPT_STATUS);
        writel(status, vm_dev->base + VIRTIO_MMIO_INTERRUPT_ACK);

puts an extra vmexit cost to trap an emulate each exit. Getting an MSI
via an exitless access to the GIC would be better I think. I'm not quite
sure what the path to IRQs from Xen is.


>
> Cheers,


-- 
Alex Bennée