RE: [Xen-devel] [Experimental PATCH] PCI and IO device emulation

["Neugebauer, Rolf" <rolf.neugebauer@xxxxxxxxx>]

> -----Original Message-----
> From: xen-devel-bounces@xxxxxxxxxxxxxxxxxxx [mailto:xen-devel-
> bounces@xxxxxxxxxxxxxxxxxxx] On Behalf Of Stefan Berger
> Sent: 30 September 2005 02:16
> To: Keir Fraser
> Cc: xen-devel@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Xen-devel] [Experimental PATCH] PCI and IO device
emulation
> 
> Keir Fraser <Keir.Fraser@xxxxxxxxxxxx> wrote on 09/28/2005 08:41:19
AM:
> 
> >
> > On 28 Sep 2005, at 03:50, Stefan Berger wrote:
> >
> > >  The attached patch is a continuation of my previous posts of
> > > *experimental* patches where I tried to slide a PCI emulation
layer
> > > underneath dom-U. This now has no more changes to the PCI code in
> > > Linux,
> > > but does all the work in Xen. This patch now also adds IO port
> > > emulation
> > > of other ports than those related to PCI. Specifically it does the
> > > following:
> >
> > I don't think this is the best approach, as it's a slippery slope.
As
> > you already note, domain0 doesn't know about hidden devices so it
can't
> > to interrupt routing and setup. Following this scheme, this would be
> > extra platform code (potentially requiring a full ACPI interpreter)
> > that would have to be added to Xen. That's a path we've already
> > considered and decided not to take.
> 
> Would it help to make the system's ACPI data available to user domains
(by
> copying them into the user domain address space upon domain creation)
so
> they can do their own translations of IRQ numbers for those devices
they
> see?

Will they come up with the same result as dom0 which sets up the IOAPICs
etc?

> 
> >
> > The only mechanism I think we should have in Xen is a protected
> > interface for allowing domU's to access PCI config space. I would
make
> > it an explicit hypercall interface rather than bothering with
emulating
> > I/O port accesses -- we have to make modifications to the PCI stack
> > anyway (otherwise we get into having to do crap like providing fake
> > BIOS tables to provide dummy bus and irq info), and adding a new
type
> > of pci read/write access method is trivial in Linux. I expect the
same
> > is true of most other OSes.
> 
> I think the dummy bus could have a potential (double) benefit for
running
> unmodified legacy OSes as well and would not just serve 'driver
domains'.

At the moment the PCI emulation for unmodified guests is dealt with in
the device model in dom0. Are you suggesting to pull this into Xen
instead?


> >
> > This interface will reject all config accesses by default, but
domain0
> > can change access privileges on a per-device basis. All that Xen has
to
> 
> 
> > do is then mask off some of the registers for write access (e.g.,
don;t
> > allow domU to arbitrarily rewrite resource base addresses) and
possibly
> > fake out reads for certain registers (e.g., perhaps the IRQ number
> > register).
> 
> At least that part you could also solve by providing a PCI emulation
> layer. Some devices can be accessible, others remain hidden.

Sure, but then you have to PCI emulation layer in Xen. Just doing what
keir suggests requires one file...

> You can also
> intercept requests to the IRQ number register and maybe return the
> system's real IRQ number without having it translated in the user
domain.

That's what Keir suggested. The problem is that at least Linux seems to
largely ignore the IRQ number in the PCI config space.

> >
> > All other smarts belong in domain0 imo. The only reason for not
doing
> > the whole lot in domain0 is that a pcifront/pciback split driver
would
> > be a lot more pain to write and to debug.
> 
> :-) I would push those smarts downwards to avoid changes in OS(es).

With Keir' proposal there are three types of changes:

- changes required to non-dom0 OSes. I think these are minimal in keir's
proposal and only require changes to arch-dependent code. Linux already
supports three PCI config probing methods on ia32. adding a 4th one
which uses a hypercall interface is trivial. The other change would be
to the IRQ setup. You'd have to convince the OS to believe the IRQ
number in the config space (or get it by some other means). I think that
is straight forward as well and both has been done in Xen 2.X. You can
even delete/remove an lot of code related to quirks in the real HW.

- changes to dom0 kernels. These are trickier in Xen 3.x. dom0 has to do
the entire setup for the device as it normally would but then not start
a device driver for devices which are supposed to be hidden from it.
This is likely to require a small change to arch-independent code in
Linux. It would also be nice if dom0 could still see all devices in the
system via things like lspci to make it easier to do device assignment

Another requirement is that both these changes need to coexist in the
same kernel so that no separate kernels are required for dom0 and domUs

- addition to the xen hypercall interface to allow domUs to probe the
PCI config space. This is not strictly required as one could just
emulate the io reads/writes of a guests, but it seems cleaner to do it
via a hypercall interface. There is also only a small addition to the
hypervisor to deal with the hypercalls. But in xen 2x this is about 

Your proposal doesn't require changes to guests but adds significantly
more code to Xen. I think the changes required to device driver domains
are small.

Could you elaborate a bit more on how your proposal interacts with Dom0
and the IRQ routing/IO APIC set up it is doing at the moment. If you
hide the device completely, as I understand you are doing, how will that
be done?

Thanks
Rolf



> 
>   Stefan
> >
> >   -- Keir
> >
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@xxxxxxxxxxxxxxxxxxx
> http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel