Re: [RFC v1 3/5] xen/arm: introduce SCMI-SMC mediator driver

[Oleksii Moisieiev <Oleksii_Moisieiev@xxxxxxxx>]

On Tue, Jan 25, 2022 at 01:19:46PM -0800, Stefano Stabellini wrote:
> On Tue, 25 Jan 2022, Oleksii Moisieiev wrote:
> > On Mon, Jan 24, 2022 at 02:14:43PM -0800, Stefano Stabellini wrote:
> > > On Mon, 24 Jan 2022, Julien Grall wrote:
> > > > On 24/01/2022 19:06, Stefano Stabellini wrote:
> > > > > It looks like XEN_DOMCTL_host_node_by_path and
> > > > > XEN_DOMCTL_find_host_compatible_node would also solve the problem but 
> > > > > I
> > > > > think that a single hypercall that retrieves the entire host DTB would
> > > > > be easier to implement
> > > > 
> > > > DOMCTL should only be used to handle per-domain information. If we want 
> > > > to
> > > > create a new sub-hypercall of either __HYPERVISOR_platform_op or
> > > > __HYPERVISOR_sysctl_op (not sure which one).
> > > > 
> > > > AFAICT, both are versioned.
> > > > 
> > > > > and more robust in the long term. >
> > > > > hypfs has the advantage that it would create an interface more similar
> > > > > to the one people are already used to on Linux systems
> > > > > (/proc/device-tree). xl/libxl would have to scan the whole hypfs tree,
> > > > > which intuitively I think it would be slower.
> > > > 
> > > > Even if you have the binary blob, you would still have to scan the
> > > > device-tree. That said, it is probably going to be potentially a bit 
> > > > faster
> > > > because you have less hypercall.
> > > > 
> > > > However, here this is a trade-off between memory use and speed. If you 
> > > > want
> > > > speed, then you may have to transfer up to 2MB every time. So the 
> > > > question is
> > > > do we care more about speed or memory usage?
> > > > 
> > > > > Also the feature might be
> > > > > harder to implement but I am not sure.
> > > > > 
> > > > > I don't have a strong preference and this is not a stable interface 
> > > > > (we
> > > > > don't have to be extra paranoid about forward and backward
> > > > > compatibility). So I am fine either way. Let's see what the others 
> > > > > think
> > > > > as well.
> > > > 
> > > > My preference would be to use hypfs as this is cleaner than exposing a 
> > > > blob.
> > > 
> > > That's also fine by me. Probably the hypfs implementation shouldn't be
> > > much more difficult than something like
> > > XEN_DOMCTL_host_node_by_path/XEN_DOMCTL_find_host_compatible_node.
> > > 
> > > 
> > > > However, are we sure we can simply copy the content of the host 
> > > > Device-Tree to
> > > > the guest Device-Tree for SCMI? For instance, I know that for device
> > > > passthrough there are some property that needs to be altered for some 
> > > > devices.
> > > > Hence, why it is not present. Although, I vaguely recalled to have 
> > > > written a
> > > > PoC, not sure if it was posted on the ML.
> > > 
> > > The SCMI node cannot be copied "as is" from host to guest. It needs a
> > > couple of changes but they seem feasible as they are limited to the
> > > channels exposed to the guest. (The generic device passthrough case is a
> > > lot more difficult.)
> > 
> > 
> > Hi Stefano,
> > 
> > What I'm thinking about is do we actually need to create SCMI node in DomU 
> > device-tree?
> > I have this question is because we don't need SCMI node to be present in 
> > DomU 
> > device-tree if it has no passed-through devices, which are using scmi. 
> > So if we don't have passed-through devices or do not provide DomU partial 
> > device-tree 
> > in config, then there is no need to create SCMI node.
> > 
> > For now I see the following possible domu configurations:
> > 1) If DomU has a lot of passed-through devices and it's easier to inherit 
> > host device-tree and disable not passed-through devices.
> > Partial device tree will looks like this:
> > 
> > #include "r8a77961-salvator-xs.dts" //include host device tree
> > 
> > /
> > {
> >     soc {
> >             ...
> >     }
> > 
> > };
> > 
> > // Disable non passed-through devices
> > &hscif {
> >     status = "disabled";
> > };
> > 
> > In this case DomU partial device-tree will inherit arm,scmi-smc and 
> > arm,scmi-shmem nodes and all clock/reset/power-domains which are using 
> > scmi. 
> > All this nodes can be copied to DomU device-tree from partial device-tree.
> 
> This is an almost dom0 configuration. For this kind of use-cases, I
> think it is enough to handle dom0 automatically correctly. I wouldn't
> ask for anything more than that.
> 
> 
> > 2) DomU has few passed-through devices, so it's easier to add the device 
> > nodes 
> > to the passthrough node of DomU partial device-tree.
> > DomU partial device-tree will look like this:
> > {
> >     scmi_shmem: scp-shmem@0x53FF0000 {
> >             compatible = "arm,scmi-shmem";
> >             reg = <0x0 0x53FF0000 0x0 0x10000>;  
> >     };
> >     scmi {
> >             arm,smc-id = <....>;
> >             compatible = "arm,scmi-smc"; 
> >             shmem = <&scmi_shmem>;
> >             scmi_clock: protocol@14 {
> >                     ...
> >             };
> >             scmi_reset: protocol@16 {
> >                     ...
> >             };
> >     }; 
> >     passthrough {
> >             hscif0: serial@e6540000 { 
> >                     compatible = "renesas,hscif-r8a77961";
> >                     scmi_devid = <5>;
> >                     clocks = <&scmi_clock 5>;
> >                     resets = <&scmi_reset 5>;
> >                     ...
> >             };
> >     };
> > };
> > 
> > As you can see in this case we have to manually copy arm,scmi-shmem and 
> > arm,scmi-smc nodes with hscif0 node or the device-tree compilation will 
> > fail.
> > We can use 0x53FF0000, provided in arm,scmi-shmem node and map domain 
> > channel 
> > to this address and copy scmi related nodes to the DomU device-tree.
> > This is useful when we need to expose only certain protocols to the DomU. 
> > Also it's easy to modify DomU scmi node, as we need for stm32mp1 for 
> > example 
> > when different smc-id should be set for DomU.
> 
> I think this is the most interesting case that should be automated and
> not require manual intervention. Let me explain why.
> 
> Currently we require partial device trees to be manually written because
> there is no easy way to automatically generate them. (I have some ideas
> on how to automatically generate partial device trees but that is a
> separate discussion.)
> 
> Unfortunately, it has become increasingly clear that it is too difficult
> for users (even advanced users!) to come up with the appropriate partial
> device trees. Thus, I am reluctant to introduce more things that rely on
> the user having to manually specify partial device tree information.
> This is why I would like the SCMI nodes to be automatically added for
> domUs.
> 
> Of course, if a user provides the scmi and scmi_shmem nodes in the
> partial device tree we could just use them. But ideally we should also
> be able to automatically generated them based on the host device tree
> nodes, so that the user only needs to provide serial@e6540000 (in your
> example, scmi_devid would need to be populated too) and the rest would
> be done automatically as we do today for the gic and vuart nodes.
> 
> At the same time I don't want to scope-creep this patch series too much
> and I don't mean to ask you to add a huge new infrastructure to Xen and
> the Xen tools just to get SCMI support in. I would rather have a
> not-great automatic generation of the domU SCMI nodes than nothing (e.g.
> using your suggested XEN_DOMCTL_host_node_by_path and
> XEN_DOMCTL_find_host_compatible_node hypercalls althought they would
> need to be platform_op as Julien suggested).
> 
> It looks like the generation of scmi_shmem and scmi should be easy
> enough that could be handled without difficulty in xl/libxl. But if that
> turns out to be too difficult, we could have a small independent
> bash/python script that from the host device tree generates the partial
> device tree with the SCMI nodes. From Xen point of view we are would
> still be using the partial device tree, but the partial device tree
> itself would be generated instead of manually written. As this workflow
> requires a separate tool I think it is a worse option than the one
> above. Still better than nothing though.
> 
Hi Stefano,

Thank you for the detail answer. I went through hypfs and will try to
export host device_tree. Then xl will be able to use hypfs data to
generate arm,scmi-smc node if arm,scmi-smc node was not provided in
DomU partial device-tree. Unfortunately, some changes should be done
to hypfs because it seems not ready to handle nested dynamic dirs.

I'll see if I can update hypfs without breaking the original
functionality. If not, I will have to create all hypfs dir structure on
start. For now I'm working on making dynamically created hypfs tree
structure, based on host device-tree.

Best regards,
Oleksii.
> 
> > 3) DomU doesn't have any passthrough nodes, which are using scmi.
> > In this case we don't want SCMI nodes to be in the DomU device-tree.
> > 
> > I see only one use-case when we may need scmi nodes to be generated by xl 
> > in 
> > DomU device-tree:
> > Xen generates psci node to handle cpu_on and cpu_off. 
> > According to the Section 4.3.2.5 of the DEN0056C [1]:
> > > For these power domains, this protocol can be used to implement PSCI 
> > > CPU_SUSPEND, CPU_ON, CPU_FREEZE, CPU_DEFAULT_SUSPEND and CPU_OFF 
> > > functions.
> > 
> > So in theory psci node can use scmi to control cpu state. But this is not 
> > our 
> > use-case because we don't want to give DomU ability to stop physical CPU. 
> > Xen can't intercept and handle CPU_ON and CPU_OFF requests when mailbox 
> > transport 
> > is used for SCMI communication.
> > 
> > [1] "SCMI Specification DEN0056C," [Online]. Available: 
> > https://urldefense.com/v3/__https://developer.arm.com/documentation/den0056/latest__;!!GF_29dbcQIUBPA!k5oB4BpbIN-hU5jrWvNy9FLXi3Kavu3qTr5lESjK8NnlS261E0Nuqg2_pUQWxb2hDdLa$
> >  [developer[.]arm[.]com] 
> 
> I agree with you on this one; I am not worried about this case.