RE: [Xen-devel] [Experimental PATCH] PCI and IO device emulation

[Stefan Berger <stefanb@xxxxxxxxxx>]

"Neugebauer, Rolf" <rolf.neugebauer@xxxxxxxxx>
wrote on 09/30/2005 05:53:36 AM:

> 
> 
> > -----Original Message-----
> > From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-
> > bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Stefan Berger
> > Sent: 30 September 2005 02:16
> > To: Keir Fraser
> > Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> > Subject: Re: [Xen-devel] [Experimental PATCH] PCI and IO device
> emulation
> > 
> > Keir Fraser <Keir.Fraser@xxxxxxxxxxxx> wrote on 09/28/2005
08:41:19
> AM:
> > 
> > >
> > > On 28 Sep 2005, at 03:50, Stefan Berger wrote:
> > >
> 
> > 
> > >
> > > The only mechanism I think we should have in Xen is a protected
> > > interface for allowing domU's to access PCI config space.
I would
> make
> > > it an explicit hypercall interface rather than bothering
with
> emulating
> > > I/O port accesses -- we have to make modifications to the
PCI stack
> > > anyway (otherwise we get into having to do crap like providing
fake
> > > BIOS tables to provide dummy bus and irq info), and adding
a new
> type
> > > of pci read/write access method is trivial in Linux. I expect
the
> same
> > > is true of most other OSes.
> > 
> > I think the dummy bus could have a potential (double) benefit
for
> running
> > unmodified legacy OSes as well and would not just serve 'driver
> domains'.
> 
> At the moment the PCI emulation for unmodified guests is dealt with
in
> the device model in dom0. Are you suggesting to pull this into Xen
> instead?

If we can come up with architecture that can serve
both purposes, enable driver domains and provide access to an emulated
PCI bus (or even real devices) for unmodified guests, then I would do it.

> 
> 
> > >
> > > This interface will reject all config accesses by default,
but
> domain0
> > > can change access privileges on a per-device basis. All
that Xen has
> to
> > 
> > 
> > > do is then mask off some of the registers for write access
(e.g.,
> don;t
> > > allow domU to arbitrarily rewrite resource base addresses)
and
> possibly
> > > fake out reads for certain registers (e.g., perhaps the
IRQ number
> > > register).
> > 
> > At least that part you could also solve by providing a PCI emulation
> > layer. Some devices can be accessible, others remain hidden.
> 
> Sure, but then you have to PCI emulation layer in Xen. Just doing
what
> keir suggests requires one file...
> 
> > You can also
> > intercept requests to the IRQ number register and maybe return
the
> > system's real IRQ number without having it translated in the
user
> domain.
> 
> That's what Keir suggested. The problem is that at least Linux seems
to
> largely ignore the IRQ number in the PCI config space.

I am saying that this can also be done in the PCI
emulation layer.

> 
> > >
> > > All other smarts belong in domain0 imo. The only reason
for not
> doing
> > > the whole lot in domain0 is that a pcifront/pciback split
driver
> would
> > > be a lot more pain to write and to debug.
> > 
> > :-) I would push those smarts downwards to avoid changes in OS(es).
> 
> With Keir' proposal there are three types of changes:
> 
> - changes required to non-dom0 OSes. I think these are minimal in
keir's
> proposal and only require changes to arch-dependent code. Linux already
> supports three PCI config probing methods on ia32. adding a 4th one
> which uses a hypercall interface is trivial. The other change would
be
> to the IRQ setup. You'd have to convince the
OS to believe the IRQ
> number in the config space (or get it by some other means). I think
that

I also think it would be important that the OS use
the IRQ number in the config space. The Hypervisor can always do translation
of IRQ numbers from machine physical to VM in the hypervisor before propagating
the IRQ to the domain.

> is straight forward as well and both has been done in Xen 2.X. You
can
> even delete/remove an lot of code related to quirks in the real HW.
>
> - changes to dom0 kernels. These are trickier
in Xen 3.x. dom0 has to do
> the entire setup for the device as it normally would but then not
start
> a device driver for devices which are supposed to be hidden from it.
> This is likely to require a small change to arch-independent code
in
> Linux. It would also be nice if dom0 could still see all devices in
the
> system via things like lspci to make it easier to do device assignment
> 
> Another requirement is that both these changes need to coexist in
the
> same kernel so that no separate kernels are required for dom0 and
domUs
> 
> - addition to the xen hypercall interface to allow domUs to probe
the
> PCI config space. This is not strictly required as one could just
> emulate the io reads/writes of a guests, but it seems cleaner to do
it
> via a hypercall interface. There is also only a small addition to
the
> hypervisor to deal with the hypercalls. But in xen 2x this is about


Access to PCI config space is protected by spinlocks.
First one writes bus/device/function and address into the config register
then one reads or writes from/to the data register affecting the previously
given address. To properly protect access to the config space it really
should be done in a common place and accessed in the same way by domain
0 and any other domain.

> 
> Your proposal doesn't require changes to guests but adds significantly
> more code to Xen. I think the changes required to device driver domains
> are small.
> 
> Could you elaborate a bit more on how your proposal interacts with
Dom0
> and the IRQ routing/IO APIC set up it is doing at the moment. If you
> hide the device completely, as I understand you are doing, how will
that
> be done?

I don't think you would like to move ownership of
the IO-APIC to Xen and let Xen do that part of the setup.  Therefore,
I would build a request list for domain 0 to do the necessary setup for
those devices it does not see and provide the HV with the necessary information
(like machine physical interrupt) it needs to properly route the IRQs to
a domain that is accessing a device. This would require an additional module
in the domain 0 kernel and some more hypercalls.

   Stefan
 
> 
> Thanks
> Rolf
> 
> 
> 
> > 
> >   Stefan
> > >
> > >   -- Keir
> > >
> > 
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@xxxxxxxxxxxxxxxxxxx
> > http://lists.xensource.com/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel