Re: [Xen-devel] [PATCH 0/3] xen: msi support for Xen dom0

[Jeremy Fitzhardinge <jeremy@xxxxxxxx>]

On 08/18/09 18:45, Qing He wrote:
> On Wed, 2009-08-19 at 04:24 +0800, Jeremy Fitzhardinge wrote:
>   
>> On 08/17/09 22:45, Qing He wrote:
>>     
>>> This patch set adds support for msi in Xen dom0. It's based on the
>>> pci notifier patches of Weidong Han (on rebase/pci branch) and
>>> contains the following 3 patches.
>>>
>>>     [PATCH 1/3] xen: make pci notifier work with booting devices
>>>     [PATCH 2/3] xen: add msi support for dom0
>>>     [PATCH 3/3] xen: re-enable msi (effectively revert bf89bc29)
>>>   
>>>       
>> Thanks, I've applied these to rebase/dom0/msi for now.   I haven't
>> tested them (or really compiled them) yet, so please look at the branch
>> and see that everything's OK.
>>
>>     
>>> One of the problem left is how to save/restore MSI across S3. Since
>>> pci_restore_msi_state() now doesn't have any arch specific hook, the
>>> code in arch/x86/ won't get a chance to run during S3 wakeup, so
>>> write_msi_msg() is called instead of xen specific functions. One of
>>> the possible solutions (and which I prefer) is to add something like
>>> arch_pci_restore_msi, but that involves slightly changing
>>> drivers/pci/msi.c, which probably needs more thinking and discussion.
>>>
>>> An alternative is to trap and emulate any access to pci configuration
>>> space. In that case, nothing in dom0 needs changing, and write_msi_msg
>>> can be reused, but considerable logic may need to change in Xen
>>> hypervisor.
>>>   
>>>       
>> The approach taken by 2/3 is not really going to fly upstream, and is
>> broadly incompatible with my intended design for interrupt handling,
>> which is to decouple the Xen/dom0 aspects of interrupt handling from the
>> apic/ioapic code entirely.  I don't know what impact this will have on
>> MSI support.  I'd appreciate it if you could look at the
>> rebase/dom0/new-interrupt-routing branch and comment on it.
>>     
> I'll have a look at it first.
>
>   
>> I'm not actually sure this approach is going to work; so far it just
>> locks up the machine shortly after ACPI initialization.  Trapping an
>> emulating (IO-)APIC accesses may well turn out to be simpler (on the
>> Linux side, at least) and more robust in the end...
>>     
> MSI by nature is vector based, but xen uses pirq as the interface to
> communicate with dom0, 2/3 is actually used to handle the pirq thing
> instead of solely vectors.
>   

Yes, that's awkward.  The new-interrupt-routing branch moves towards
eliminating vectors from the kernel<->xen, replacing them entirely with
pirqs.  Part of the motivation for this is to insulate the kernel from
Xen's decisions on how to allocate and route vectors (ie, the number of
vectors and whether they're percpu or not).

Extending this to MSI would presumably require Xen to do the actual PCI
(config space?) programming given a pirq for the device interrupt. 
Perhaps this could be best achieved by making the x86
arch_setup_msi_irqs call through a function pointer so we can direct it
to a Xen-specific version.  Would that be clean?

> So trapping and emulating for msi will first need a decoupling of that,
> allowing dom0 to use only vector to index its msi. This requires changing
> of the current pirq based xen irq, which seems quite annoying at Xen side,
> especially if per-cpu vector is in.
>   

What about making pirq == vector, so the "vector" seen by the kernel
isn't a real vector at all, and Xen remaps from the pirq/vector to the
real vector when emulating the write?

    J

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel