Re: [PATCH 2/7] x86/wait: prevent duplicated assembly labels

[Jan Beulich <jbeulich@xxxxxxxx>]

On 14.03.2025 10:05, Andrew Cooper wrote:
> On 14/03/2025 8:44 am, Jan Beulich wrote:
>> On 14.03.2025 09:30, Roger Pau Monné wrote:
>>> On Fri, Mar 14, 2025 at 09:24:09AM +0100, Jan Beulich wrote:
>>>> On 13.03.2025 16:30, Roger Pau Monne wrote:
>>>>> When enabling UBSAN with clang, the following error is triggered during 
>>>>> the
>>>>> build:
>>>>>
>>>>> common/wait.c:154:9: error: symbol '.L_wq_resume' is already defined
>>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>>       |         ^
>>>>> <inline asm>:1:121: note: instantiated into assembly here
>>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; 
>>>>> push %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov 
>>>>> %rsp,%rsi;.L_wq_resume: rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop 
>>>>> %r14; pop %r13;pop %r12; pop %rbp; pop %rbx
>>>>>       |                                                                   
>>>>>                                                              ^
>>>>> common/wait.c:154:9: error: symbol '.L_skip' is already defined
>>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>>       |         ^
>>>>> <inline asm>:1:159: note: instantiated into assembly here
>>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; 
>>>>> push %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov 
>>>>> %rsp,%rsi;.L_wq_resume: rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop 
>>>>> %r14; pop %r13;pop %r12; pop %rbp; pop %rbx
>>>>>       |                                                                   
>>>>>                                                                           
>>>>>                          ^
>>>>> 2 errors generated.
>>>>>
>>>>> The inline assembly block in __prepare_to_wait() is duplicated, thus
>>>>> leading to multiple definitions of the otherwise unique labels inside the
>>>>> assembly block.  GCC extended-asm documentation notes the possibility of
>>>>> duplicating asm blocks:
>>>>>
>>>>>> Under certain circumstances, GCC may duplicate (or remove duplicates of)
>>>>>> your assembly code when optimizing. This can lead to unexpected duplicate
>>>>>> symbol errors during compilation if your asm code defines symbols or
>>>>>> labels. Using ‘%=’ (see AssemblerTemplate) may help resolve this problem.
>>>>> Move the assembly blocks that deal with saving and restoring the current
>>>>> CPU context into it's own explicitly non-inline functions.  This prevents
>>>>> clang from duplicating the assembly blocks.  Just using noinline attribute
>>>>> seems to be enough to prevent assembly duplication, in the future noclone
>>>>> might also be required if asm block duplication issues arise again.
>>>> Wouldn't it be a far easier / less intrusive change to simply append %= to
>>>> the label names?
>>> That won't work AFAICT, as the inline asm in check_wakeup_from_wait()
>>> won't be able to make a jump to the .L_wq_resume label defined in the
>>> __prepare_to_wait() assembly block if the label is declared as
>>> .L_wq_resume%=.
>>>
>>> Also we want to make sure there's a single .L_wq_resume seeing how
>>> check_wakeup_from_wait() uses it as the restore entry point?
>> Hmm, yes on both points; the %= would only work for .Lskip. Have you gained
>> understanding why there is this duplication? The breaking out of the asm()
>> that you do isn't going to be reliable, as in principle the compiler is
>> still permitted to duplicate stuff. Afaict the only reliable way is to move
>> the code to a separate assembly file (with the asm() merely JMPing there,
>> providing a pseudo-return-address by some custom means). Or to a file-scope
>> asm(), as those can't be duplicated.
> 
> See the simplified example in
> https://github.com/llvm/llvm-project/issues/92161
> 
> When I debugged this a while back, The multiple uses of wqv->esp (one
> explicit after the asm, one as an asm parameter) gain pointer
> sanitisation, so the structure looks like:
> 
>     ...
>     if ( bad pointer )
>         __ubsan_report();
>     asm volatile (...);
>     if ( bad pointer )
>         __ubsan_report();
>     ...
> 
> which then got transformed to:
> 
>     if ( bad pointer )
>     {
>         __ubsan_report();
>         asm volatile (...);
>         __ubsan_report();
>     }
>     else
>         asm volatile (...);

But isn't it then going to be enough to latch &wqv->esp into a local variable,
and use that in the asm() and in the subsequent if()?

Jan