Re: Proposal for virtual IOMMU binding b/w vIOMMU and passthrough devices

[Julien Grall <julien@xxxxxxx>]

Hi Stefano,

On 30/10/2022 14:23, Stefano Stabellini wrote:
> On Fri, 28 Oct 2022, Julien Grall wrote:
> > On 28/10/2022 14:13, Bertrand Marquis wrote:
> > > > On 28 Oct 2022, at 14:06, Julien Grall <julien@xxxxxxx> wrote:
> > > >
> > > > Hi Rahul,
> > > >
> > > > On 28/10/2022 13:54, Rahul Singh wrote:
> > > > > > > > > > For ACPI, I would have expected the information to be found in
> > > > > > > > > > the IOREQ.
> > > > > > > > > >
> > > > > > > > > > So can you add more context why this is necessary for
> > > > > > > > > > everyone?
> > > > > > > > > We have information for IOMMU and Master-ID but we don’t have
> > > > > > > > > information for linking vMaster-ID to pMaster-ID.
> > > > > > > > I am confused. Below, you are making the virtual master ID
> > > > > > > > optional. So shouldn't this be mandatory if you really need the
> > > > > > > > mapping with the virtual ID?
> > > > > > > vMasterID is optional if user knows pMasterID is unique on the
> > > > > > > system. But if pMasterId is not unique then user needs to provide
> > > > > > > the vMasterID.
> > > > > > So the expectation is the user will be able to know that the pMasterID
> > > > > > is uniq. This may be easy with a couple of SMMUs, but if you have 50+
> > > > > > (as suggested above). This will become a pain on larger system.
> > > > > >
> > > > > > IHMO, it would be much better if we can detect that in libxl (see
> > > > > > below).
> > > > > We can make the vMasterID compulsory to avoid complexity in libxl to
> > > > > solve this
> > > > In general, complexity in libxl is not too much of problem.
> I agree with this and also I agree with Julien's other statement:
>
> "I am strongly in favor of libxl to modify it if it greatly improves the
> user experience."
>
> I am always in favor of reducing complexity for the user as they
> typically can't deal with tricky details such as MasterIDs. In general,
> I think we need more automation with our tooling.
>
> However, it might not be as simple as adding support for automatically
> generating IDs in libxl because we have 2 additional cases to support:
> 1) dom0less
> 2) statically built guests
>
> For 1) we would need the same support also in Xen? Which means more
> complexity in Xen.
Xen will need to parse the device-tree to find the mapping. So I am not 
entirely convinced there will be more complexity needed other than 
requiring a bitmap to know which vMasterID has been allocated.
That said, you would still need one to validate the input provided by 
the user. So overall maybe there will be no added complexity?
> 2) are guests like Zephyr that consume a device tree at
> build time instead of runtime. These guests are built specifically for a
> given environment and it is not a problem to rebuild them for every Xen
> release.
>
> However I think it is going to be a problem if we have to run libxl to
> get the device tree needed for the Zephyr build. That is because it
> means that the Zephyr build system would have to learn how to compile
> (or crosscompile) libxl in order to retrieve the data needed for its
> input. Even for systems based on Yocto (Yocto already knows how to build
> libxl) would cause issues because of internal dependencies this would
> introduce.
That would not be very different to how this works today for Zephyr. 
They need libxl to generate the guest DT.
That said, I agree this is a bit of a pain...

> So I think the automatic generation might be best done in another tool.
It sounds like what you want is creating something similar to libacpi 
but for Device-Tree. That should work with some caveats.
> I think we need something like a script that takes a partial device tree
> as input and provides a more detailed partial device tree as output with
> the generated IDs.
AFAICT, having the partial device-tree is not enough. You also need the 
real DT to figure out the pMaster-ID.
> If we did it that way, we could call the script from libxl, but also we
> could call it separately from ImageBuilder for dom0less and Zephyr/Yocto
> could also call it.
>
> Basically we make it easier for everyone to use it. The only price to
> pay is that it will be a bit less efficient for xl guests (one more
> script to fork and exec) but I think is a good compromise.
We would need an hypercall to retrieve the host Device-Tree. But that 
would not be too difficult to add.
[...]

> I think this is a great idea, I only suggest that we move the automatic
> generation out of libxl (a separate stand-alone script), in another
> place that can be more easily reused by multiple projects and different
> use-cases.
If we use the concept of libacpi, we may not need a to have a 
stand-alone script. It could directly linked in libxl or any other tools.
Cheers,

--
Julien Grall