Re: [Xen-devel] [PATCH v4 10/11] public: add XENFEAT_ARM_SMCCC_supported feature

[Julien Grall <julien.grall@xxxxxxxxx>]

Hello,

Sorry for the formatting, writing from my phone. Ki

On Thu, 31 Aug 2017, 22:18 Sergej Proskurin <proskurin@xxxxxxxxxxxxx> wrote:

Hi Volodymyr,


On 08/31/2017 04:58 PM, Volodymyr Babchuk wrote:
> Hi Sergej
>
> On 31.08.17 16:51, Sergej Proskurin wrote:
>> Hi Volodymyr,
>>
>>
>> On 08/31/2017 02:44 PM, Volodymyr Babchuk wrote:
>>> Hello Sergej,
>>>
>>> On 31.08.17 15:20, Sergej Proskurin wrote:
>>>> Hi Volodymyr, hi Julien,
>>>>
>>>>
>>>> On 08/24/2017 07:25 PM, Julien Grall wrote:
>>>>>
>>>>>
>>>>> On 21/08/17 21:27, Volodymyr Babchuk wrote:
>>>>>> This feature indicates that hypervisor is compatible with ARM
>>>>>> SMC calling convention. Hypervisor will not inject an undefined
>>>>>> instruction exception if an invalid SMC function were called and
>>>>>> will not crash a domain if an invlalid HVC functions were called.
>>>>>
>>>>> s/invlalid/invalid/
>>>>>
>>>>> The last sentence is misleading. Xen will still inject and undefined
>>>>> instruction for some SMC/HVC. You may want to rework it to make it
>>>>> clear.
>>>>>
>>>>
>>>> Now that you say that Xen will still inject an undefined instruction
>>>> exception for some SMCs, I have a to ask for which exactly?
>>> For ones that are compatible with ARM SMCCC [1]. E.g if you are
>>> running SMCCC-compatible system and you are calling SMC/HVC with
>>> immediate value 0, then you are safe.
>>>
>>
>> Alright, as far as I understand this is exactly what I do right now. I
>> inject an SMC that is encoded as 0xD4000003.
> Actually, this patch series are not merged yet, so no SMCCC support
> right. But this should not a problem in your case.
>
>>>> I might be off topic here, so please tell me if you believe this is
>>>> not
>>>> the right place for this question. In this case I will open an new
>>>> thread. Right now, I am working with the previous implementation of
>>>> do_trap_smc that was extended in this patch. Yet, as far as I
>>>> understand, the behavior should not change, which is why I am asking
>>>> this quesiton in this thread.
>>> If you are talking about forwarding SMC exception to VM monitor, then
>>> yes, that should not change.
>>
>> Yes, exactly. Sorry, I forgot to mention that I have a modified
>> xen-access version running in dom0 that registers an SMC monitor and
>> also increases the PC by 4 (or dependent on the case, simply leaves it
>> as it is) on every SMC trap.
> Aha, I see. I never was able to test this feature fully. I played with
> my own VM monitor, when I tried to offload SMC handling to another
> domain. But I had to comment out most of the VM monitor code in XEN.
>
>>>
>>>> Currently, I am working on SMC guest injections and trying to
>>>> understand
>>>> the resulting behavior. Every time, right after the execution of an
>>>> injected SMC instruction, the guest traps into the undefined
>>>> instruction
>>>> exception handler in EL1 and I simply don't understand why. As far
>>>> as I
>>>> understand, as soon an injected SMC instruction gets executed, it
>>>> should
>>>> _transparently_ trap into the hypervisor (assuming MDCR_EL2.TDE is
>>>> set).
>>>> As soon as the hypervisor returns (e.g. to PC+4 or to the trapping PC
>>>> that now contains the original instruction instead of the injected
>>>> SMC),
>>>> the guest should simply continue its execution.
>>> Hm. What do you mean under "SMC instruction injection?".
>>
>> My code runs in dom0 and "injects" an SMC instruction to predefined
>> addresses inside the guest as to simulate software breakpoints. By this,
>> I mean that the code replaces the original guest instruction at a
>> certain address with an SMC. Think of a debugger that uses software
>> breakpoints. The idea is to put back the original instruction right
>> after the SMC gets called, so that the guest can continue with its
>> execution. You can find more information about that in [0], yet please
>> consider that I try to trap the SMC directly in Xen instead of
>> TrustZone.
> Yep, I see. Immediate question: do you flush icache after you put
> original instruction back?

Yeap. But the current behavior does not let me to go this far, as I the
system jumps into the interrupt handler and single-steps the handler
instead of the instruction of interest.

On your first mail, you started with "smc injection doesn't work", then "I replace instruction" and now you mention about single-stepping.

This doesn't help at all to understand what you are doing and really not related to this thread.

So can you please details exactly what you are doing rather than giving bits by bits?

> Then I can't see, why this should not work. If only VM monitor core in
> XEN is not broken. I don't know any users of this.
> I'm just curious, why are you using SMC, not BRK instruction?
>

I use SMC instructions as the guest can register for BRK events. The
guest cannot register for SMC events. So, in order stay stealthy towards
the guest and also not to cope with BRK re-injections, SMC's seemed to
be the right choice : 

I have already said that using SMC is a pretty bad idea when Tamas added the trapping and you guys still seem to think it is a good idea...

>>> Current code in hypervisor will always inject undefined instruction
>>> exception when you  call SMC (unless you installed VM monitor for the
>>> guest). Also, it will not increase PC. So, if you'll try to remove
>>> inject_undef_exception() call, you'll get into an infinite loop.
>>>
>>
>> I have a registered SMC monitor running in dom0 that does not reinject
>> the undefined instruction exception in do_trap_smc(). So there is no
>> indefinite loop at this point. What I see is that as soon as my code in
>> xen-access (dom0) increments the trapped guest PC by 4 (and also if it
>> doesn't) the next instruction inside the guest will be inside the undef
>> instruction handler (I can see that because I have implemented a single
>> stepping mechanism for AArch64 in Xen that gets activated right after
>> the guest executes the injected SMC instruction).
> That's strange. Can you print whole vCPU state to determine that PC
> points to the right place? Also you can check DFAR. Probably you can
> even dump memory pointed by DFAR to make sure that you written back
> correct instruction.

Yea, I do that. And both the SMC injection, as well as further vCPU
state seems to be correct at this point.

Today, I saw an interesting behavior in my single-stepping
implementation, which is the reason for my late reply. I can't explain
what is going wrong, yet. So I will need to further investigate this
behavior and post and RFC for the single-stepping mechanism as to put
more eyes on the issue. Maybe, this will help solve it.

But anyway, thank you very much for your help! I really appreciate it :)

You probably want to look at https://lists.xen.org/archives/html/xen-devel/2017-08/msg00661.html and maybe sync-up with this person if you are not working with him.

Cheers,

>
>>>> Now, according to ARM DDI0487B.a D1-1873, the following holds: "If
>>>> HCR_EL2.TSC or HCR.TSC traps attempted EL1 execution of SMC
>>>> instructions
>>>> to EL2, that trap has priority over this disable". So this means
>>>> that if
>>>> SMCs are disabled for NS EL1, the guest will trap into the
>>>> hypervisor on
>>>> SMC execution. Yet, since SMCs are disabled from NS EL1, the guest
>>>> will
>>>> execute an undefined instrcution exception. Which is what I was
>>>> thinking
>>>> about is currently happening on my ARMv8 dev board (Lemaker Hikey). On
>>>> the other hand I believe that it is highly unlikely that the EFI
>>>> loader
>>>> explicitly disables SMC's for NS EL1. However, since I don't have
>>>> access
>>>> to SCR_EL3.SMD from EL2, I can't tell whether this is the reason
>>>> for the
>>>> behavior I am experiencing on my board or not.
>>> According to ARM ARM, hypervisor should trap SMC even if was disabled
>>> by EL3. I think, that in your case the problem is in current
>>> implementation of do_trap_smc()
>>>
>>
>> Unfortunately, I don't think that this is the problem of do_trap_smc()
>> (see above). But let me check one more time.
> As I said, I know no users for SMC monitor, and I'm not exactly sure
> that it works properly.
>
>>
>>>> It would be of great help if you would provide me with some more
>>>> clarity
>>>> on my case. I am sure that I have missed something that simply needs
>>>> clarification. Thank you very much in advance.
>>> I don't quite understood, what you are trying to achieve. But I think
>>> that pair of printk()s in do_trap_smc() will reveal much.
>>>
>>
>> Yea, the idea is to inject SMC instructions into the guest to simulate
>> software breakpoints for guest analysis purposes. Please let me cleanup
>> my current printk output to better present my issue.
>>
>>>
>>> [1]
>>> http://infocenter.arm.com/help/topic/com.arm.doc.den0028b/ARM_DEN0028B_SMC_Calling_Convention.pdf
>>>
>>
>> Thank you,
>> ~Sergej
>>
>> [0] http://www.cse.psu.edu/~trj1/papers/most14.pdf
>>

Thanks,
~Sergej


_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@xxxxxxxxxxxxx
https://lists.xen.org/xen-devel