Re: OS Headers hypercall.h/hypervisor.h

[Julien Grall <julien@xxxxxxx>]

On 31/03/2022 09:44, Julien Grall wrote:
> On 31/03/2022 02:15, Elliott Mitchell wrote:
> > On Tue, Mar 29, 2022 at 08:27:24AM +0200, Jan Beulich wrote:
> > > On 29.03.2022 00:25, Stefano Stabellini wrote:
> > > > On Sat, 26 Mar 2022, Elliott Mitchell wrote:
> > > > > The hypercalls implementation for Linux and FreeBSD have two key 
> > > > > headers,
> > > > > hypercall.h and hypervisor.h.  I'm curious why the implementations for
> > > > > x86 and ARM* are so distinct.
> > > > >
> > > > > I found it fairly straightforward to implement ARM* versions of the 
> > > > > x86
> > > > > _hypercall#() macros.  Once that is done, most of the wrappers in 
> > > > > the x86
> > > > > hypercall.h can be moved to a shared hypervisor.h header.
> > > > >
> > > > > Why does Xen/ARM on Linux still have hypercall.S when merging the
> > > > > headers should reduce maintainance?
> > > > >
> > > > > Was GCC extended inline assembly language for ARM* thought too awful?
> > > > >
> > > > > I'm also curious why these headers are part of the Linux kernel, 
> > > > > instead
> > > > > of being maintained by the Xen Project?
> > > > I would have to dig through ancient archives to give you a full answer
> > > > but the reason was that the asm inline on ARM didn't provide enough
> > > > guarantees on ordering and registers it would use and clobber.
> > > While there may be ordering issues (albeit most ought to be taken care
> > > of by marking the asm() volatile and having it have a memory clobber),
> > > registers used / clobbered ought to always be expressable by asm() -
> > > if not by constraints covering just a single register (such frequently
> > > simply don't exist), then by using register variables tied to a
> > > particular register. Of course in all of this there's an assumption of
> > > no bugs in this area in the compilers claimed as being supported ...
> > I'm merely been working with recent versions of Clang on FreeBSD, but
> > I've got something which appears to work.
> >
> > I would be somewhat hopeful GCC might have fewer bugs on ARM as
> > registers aren't so precious.  Yet that could well be a minefield.
> Linux and Xen have already some code heavily using inline assembly for 
> the SMCCC v1.1 helpers. So, in theory, it should be possible to convert 
> the hypercall to use inline assembly helpers.
> Unlike SMCCC v1.1, the hypercalls are following the AAPCS. So by using 
> the assembly wrapper we don't have to worry on what to save as the 
> compiler will automatically know what to do. Looking at 
> xen/include/public/arch-arm.h, we may be able to reduce the numbers of 
> registers to preserve. So it would be more interesting to switch to 
> inline assembly to reduce the number of instructions.
> That said, we also need to be mindful of straigh-line speculation with 
> HVC instruction. With the inline version, the speculation barrier (sb or 
> dsb/isb) would need to be architecturally executed. With the assembly 
> wrapper, I believe we could only speculatively execute it by adding 
> after the ret.
Please ignore this bit. Digging through my e-mails, I found the private 
discussion regarding this mitigation. Below what I wrote back in 2020:
"The barrier-after-RET is only here to protect against straight-line
speculation of the RET instruction. Not the SMC.

If you have a processor that will always do straight-line speculation
for both the SMC and the RET, then yes the barrier afterwards will block
further speculation.

But let's imagine you have an hypothetical processor that only do
straight-line speculation for the SMC. For the RET instruction, it will
instead decide to speculatively fetch and execute the instructions
pointed out by LR.

This means that the barrier after the RET would never get fetched or
executed speculatively. In this case, there would be no barrier to
speculate even further and reach a potential reveal gadget."

So if we need to mitigate the straight-line-speculation, we would need a 
speculation barrier right after SMC/HVC regardless the approach.
I will have a look at the header Elliott shared.

Cheers,

--
Julien Grall