Re: [PATCH 2/7] x86/wait: prevent duplicated assembly labels

[Andrew Cooper <andrew.cooper3@xxxxxxxxxx>]

On 14/03/2025 8:44 am, Jan Beulich wrote:
> On 14.03.2025 09:30, Roger Pau Monné wrote:
>> On Fri, Mar 14, 2025 at 09:24:09AM +0100, Jan Beulich wrote:
>>> On 13.03.2025 16:30, Roger Pau Monne wrote:
>>>> When enabling UBSAN with clang, the following error is triggered during the
>>>> build:
>>>>
>>>> common/wait.c:154:9: error: symbol '.L_wq_resume' is already defined
>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>       |         ^
>>>> <inline asm>:1:121: note: instantiated into assembly here
>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; push 
>>>> %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov %rsp,%rsi;.L_wq_resume: 
>>>> rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop %r14; pop %r13;pop %r12; pop 
>>>> %rbp; pop %rbx
>>>>       |                                                                    
>>>>                                                             ^
>>>> common/wait.c:154:9: error: symbol '.L_skip' is already defined
>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>       |         ^
>>>> <inline asm>:1:159: note: instantiated into assembly here
>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; push 
>>>> %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov %rsp,%rsi;.L_wq_resume: 
>>>> rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop %r14; pop %r13;pop %r12; pop 
>>>> %rbp; pop %rbx
>>>>       |                                                                    
>>>>                                                                            
>>>>                        ^
>>>> 2 errors generated.
>>>>
>>>> The inline assembly block in __prepare_to_wait() is duplicated, thus
>>>> leading to multiple definitions of the otherwise unique labels inside the
>>>> assembly block.  GCC extended-asm documentation notes the possibility of
>>>> duplicating asm blocks:
>>>>
>>>>> Under certain circumstances, GCC may duplicate (or remove duplicates of)
>>>>> your assembly code when optimizing. This can lead to unexpected duplicate
>>>>> symbol errors during compilation if your asm code defines symbols or
>>>>> labels. Using ‘%=’ (see AssemblerTemplate) may help resolve this problem.
>>>> Move the assembly blocks that deal with saving and restoring the current
>>>> CPU context into it's own explicitly non-inline functions.  This prevents
>>>> clang from duplicating the assembly blocks.  Just using noinline attribute
>>>> seems to be enough to prevent assembly duplication, in the future noclone
>>>> might also be required if asm block duplication issues arise again.
>>> Wouldn't it be a far easier / less intrusive change to simply append %= to
>>> the label names?
>> That won't work AFAICT, as the inline asm in check_wakeup_from_wait()
>> won't be able to make a jump to the .L_wq_resume label defined in the
>> __prepare_to_wait() assembly block if the label is declared as
>> .L_wq_resume%=.
>>
>> Also we want to make sure there's a single .L_wq_resume seeing how
>> check_wakeup_from_wait() uses it as the restore entry point?
> Hmm, yes on both points; the %= would only work for .Lskip. Have you gained
> understanding why there is this duplication? The breaking out of the asm()
> that you do isn't going to be reliable, as in principle the compiler is
> still permitted to duplicate stuff. Afaict the only reliable way is to move
> the code to a separate assembly file (with the asm() merely JMPing there,
> providing a pseudo-return-address by some custom means). Or to a file-scope
> asm(), as those can't be duplicated.

See the simplified example in
https://github.com/llvm/llvm-project/issues/92161

When I debugged this a while back, The multiple uses of wqv->esp (one
explicit after the asm, one as an asm parameter) gain pointer
sanitisation, so the structure looks like:

    ...
    if ( bad pointer )
        __ubsan_report();
    asm volatile (...);
    if ( bad pointer )
        __ubsan_report();
    ...

which then got transformed to:

    if ( bad pointer )
    {
        __ubsan_report();
        asm volatile (...);
        __ubsan_report();
    }
    else
        asm volatile (...);

~Andrew