Re: [Xen-devel] [rfc 00/18] ioemu: use devfn instead of slots as the unit for passthrough

[Simon Horman <horms@xxxxxxxxxxxx>]

On Thu, Mar 05, 2009 at 08:34:57AM +0000, Keir Fraser wrote:
> On 04/03/2009 23:05, "Simon Horman" <horms@xxxxxxxxxxxx> wrote:
> 
> > I am guessing that your idea was not for xend to allocate all
> > of those devfn and pass them to qemu-dm on the command line.
> 
> That would be reasonable. Does xend need to know or care about the virtual
> devfn for any reason?

I don't think that xend cares about devfn at this time.

> > My instinct is that it would really be easier to allocate devn -
> > using something like the algorithm I describe below - in qemu-dm
> > rather than xend.
> 
> The python version looks unfeasibly complicated. Perhaps this kind of thing
> is easier implemented in C? :-)

I should note that was the first python code I have written
(other than the occasional bug fix here and there). I expect
that I could write a more reasonable implementation in C without
too much trouble. (I also expect I could write a more reasonable
python version if needed.)

Another reason that the code is complex is that the mapping
rules - expercially regarding sharing of GSI - seem complex to me.
And to be honest, I'm not sure that I have them correct yet.

Currently my assumptions are:
* ioemu devices may share GSI, though its best to limit sharing as
  much as possible
* pass-through devices may not share a GSI with another pass-through
  device, and its better if they don't share a GSI with an ioemu
  device either
* for multi-function devices, the device needs 4 consecutive GSI,
  so that INTA, B, C and D can be used by functions of the device.
  - this can probably be relaxed in many cases as its probably
    true that many multi-function devices don't use all of
    INTA, B, C and D. But my feeling is that accounting
    for this will add complexity rather than remove it.

* I have not accounted for MSI devices/functions at this time,
  but my understanding is that they don't need a GSI at all.
  So it should be just a matter of giving them a device
  and making sure nothing else uses it.


A question related to GSI that I would like your opinion on is
if devices need to be allocated sequentially. Currently the
code that allocates devfn (which is in qemu-dm to answer your
question from another email) tries to allocate them sequentially,
save a hole left for hotplug.

After studying the diagram that Shimada-san sent, I have concluded that in
order to reduce fragmentation of the GSI space (so that 4 continous GSI can
be allocated to multi-function devices as much as possible, assuming that
the number of pass-through devices is expanded from 2, which I would like
to do) it would be better to allocate every 8th device, which corresponds
to using intA for each GSI in turn.

GSI 16, INTA -> Dev 0
GSI 17, INTA -> Dev 8
GSI 18, INTA -> Dev 16
GSI 19, INTA -> Dev 24
GSI 20, INTA -> Dev 1
...

So instead of allocating by scanning the rows in the diagram,
we work down the columns.

http://lists.xensource.com/archives/html/xen-devel/2009-02/pngmIO1Sm0VEX.png

Or to give a more conrete example:
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)
00: 4.0 Ethernet controller: Realtek Semiconductor Co., Ltd.  
RTL-8139/8139C/8139C+ (rev 20)

Would become:
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 Ethernet controller: Realtek Semiconductor Co., Ltd.  
RTL-8139/8139C/8139C+ (rev 20)
00:08.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:08.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:08.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:16.0 VGA compatible controller: Cirrus Logic GD 5446
00:24.0 Class ff80: XenSource, Inc. Xen Platform Device (rev 01)

I can't actually get a system with this mapping to boot
(which prehaps means its flawed for the first few devices).
But a fundamental problem (e.g. for users) with using
such an allocation stratergy?

-- 
Simon Horman
  VA Linux Systems Japan K.K., Sydney, Australia Satellite Office
  H: www.vergenet.net/~horms/             W: www.valinux.co.jp/en


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-devel