[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH 2/7] x86/wait: prevent duplicated assembly labels


  • To: Roger Pau Monné <roger.pau@xxxxxxxxxx>
  • From: Jan Beulich <jbeulich@xxxxxxxx>
  • Date: Fri, 14 Mar 2025 12:17:39 +0100
  • Autocrypt: addr=jbeulich@xxxxxxxx; keydata= xsDiBFk3nEQRBADAEaSw6zC/EJkiwGPXbWtPxl2xCdSoeepS07jW8UgcHNurfHvUzogEq5xk hu507c3BarVjyWCJOylMNR98Yd8VqD9UfmX0Hb8/BrA+Hl6/DB/eqGptrf4BSRwcZQM32aZK 7Pj2XbGWIUrZrd70x1eAP9QE3P79Y2oLrsCgbZJfEwCgvz9JjGmQqQkRiTVzlZVCJYcyGGsD /0tbFCzD2h20ahe8rC1gbb3K3qk+LpBtvjBu1RY9drYk0NymiGbJWZgab6t1jM7sk2vuf0Py O9Hf9XBmK0uE9IgMaiCpc32XV9oASz6UJebwkX+zF2jG5I1BfnO9g7KlotcA/v5ClMjgo6Gl MDY4HxoSRu3i1cqqSDtVlt+AOVBJBACrZcnHAUSuCXBPy0jOlBhxPqRWv6ND4c9PH1xjQ3NP nxJuMBS8rnNg22uyfAgmBKNLpLgAGVRMZGaGoJObGf72s6TeIqKJo/LtggAS9qAUiuKVnygo 3wjfkS9A3DRO+SpU7JqWdsveeIQyeyEJ/8PTowmSQLakF+3fote9ybzd880fSmFuIEJldWxp Y2ggPGpiZXVsaWNoQHN1c2UuY29tPsJgBBMRAgAgBQJZN5xEAhsDBgsJCAcDAgQVAggDBBYC AwECHgECF4AACgkQoDSui/t3IH4J+wCfQ5jHdEjCRHj23O/5ttg9r9OIruwAn3103WUITZee e7Sbg12UgcQ5lv7SzsFNBFk3nEQQCACCuTjCjFOUdi5Nm244F+78kLghRcin/awv+IrTcIWF hUpSs1Y91iQQ7KItirz5uwCPlwejSJDQJLIS+QtJHaXDXeV6NI0Uef1hP20+y8qydDiVkv6l IreXjTb7DvksRgJNvCkWtYnlS3mYvQ9NzS9PhyALWbXnH6sIJd2O9lKS1Mrfq+y0IXCP10eS FFGg+Av3IQeFatkJAyju0PPthyTqxSI4lZYuJVPknzgaeuJv/2NccrPvmeDg6Coe7ZIeQ8Yj t0ARxu2xytAkkLCel1Lz1WLmwLstV30g80nkgZf/wr+/BXJW/oIvRlonUkxv+IbBM3dX2OV8 AmRv1ySWPTP7AAMFB/9PQK/VtlNUJvg8GXj9ootzrteGfVZVVT4XBJkfwBcpC/XcPzldjv+3 HYudvpdNK3lLujXeA5fLOH+Z/G9WBc5pFVSMocI71I8bT8lIAzreg0WvkWg5V2WZsUMlnDL9 mpwIGFhlbM3gfDMs7MPMu8YQRFVdUvtSpaAs8OFfGQ0ia3LGZcjA6Ik2+xcqscEJzNH+qh8V m5jjp28yZgaqTaRbg3M/+MTbMpicpZuqF4rnB0AQD12/3BNWDR6bmh+EkYSMcEIpQmBM51qM EKYTQGybRCjpnKHGOxG0rfFY1085mBDZCH5Kx0cl0HVJuQKC+dV2ZY5AqjcKwAxpE75MLFkr wkkEGBECAAkFAlk3nEQCGwwACgkQoDSui/t3IH7nnwCfcJWUDUFKdCsBH/E5d+0ZnMQi+G0A nAuWpQkjM1ASeQwSHEeAWPgskBQL
  • Cc: Andrew Cooper <andrew.cooper3@xxxxxxxxxx>, Anthony PERARD <anthony.perard@xxxxxxxxxx>, Michal Orzel <michal.orzel@xxxxxxx>, Julien Grall <julien@xxxxxxx>, Stefano Stabellini <sstabellini@xxxxxxxxxx>, xen-devel@xxxxxxxxxxxxxxxxxxxx
  • Delivery-date: Fri, 14 Mar 2025 11:17:46 +0000
  • List-id: Xen developer discussion <xen-devel.lists.xenproject.org>

On 14.03.2025 11:12, Roger Pau Monné wrote:
> On Fri, Mar 14, 2025 at 10:13:07AM +0100, Jan Beulich wrote:
>> On 14.03.2025 10:05, Andrew Cooper wrote:
>>> On 14/03/2025 8:44 am, Jan Beulich wrote:
>>>> On 14.03.2025 09:30, Roger Pau Monné wrote:
>>>>> On Fri, Mar 14, 2025 at 09:24:09AM +0100, Jan Beulich wrote:
>>>>>> On 13.03.2025 16:30, Roger Pau Monne wrote:
>>>>>>> When enabling UBSAN with clang, the following error is triggered during 
>>>>>>> the
>>>>>>> build:
>>>>>>>
>>>>>>> common/wait.c:154:9: error: symbol '.L_wq_resume' is already defined
>>>>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>>>>       |         ^
>>>>>>> <inline asm>:1:121: note: instantiated into assembly here
>>>>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; 
>>>>>>> push %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov 
>>>>>>> %rsp,%rsi;.L_wq_resume: rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop 
>>>>>>> %r14; pop %r13;pop %r12; pop %rbp; pop %rbx
>>>>>>>       |                                                                 
>>>>>>>                                                                ^
>>>>>>> common/wait.c:154:9: error: symbol '.L_skip' is already defined
>>>>>>>   154 |         "push %%rbx; push %%rbp; push %%r12;"
>>>>>>>       |         ^
>>>>>>> <inline asm>:1:159: note: instantiated into assembly here
>>>>>>>     1 |         push %rbx; push %rbp; push %r12;push %r13; push %r14; 
>>>>>>> push %r15;sub %esp,%ecx;cmp $4096, %ecx;ja .L_skip;mov 
>>>>>>> %rsp,%rsi;.L_wq_resume: rep movsb;mov %rsp,%rsi;.L_skip:pop %r15; pop 
>>>>>>> %r14; pop %r13;pop %r12; pop %rbp; pop %rbx
>>>>>>>       |                                                                 
>>>>>>>                                                                         
>>>>>>>                              ^
>>>>>>> 2 errors generated.
>>>>>>>
>>>>>>> The inline assembly block in __prepare_to_wait() is duplicated, thus
>>>>>>> leading to multiple definitions of the otherwise unique labels inside 
>>>>>>> the
>>>>>>> assembly block.  GCC extended-asm documentation notes the possibility of
>>>>>>> duplicating asm blocks:
>>>>>>>
>>>>>>>> Under certain circumstances, GCC may duplicate (or remove duplicates 
>>>>>>>> of)
>>>>>>>> your assembly code when optimizing. This can lead to unexpected 
>>>>>>>> duplicate
>>>>>>>> symbol errors during compilation if your asm code defines symbols or
>>>>>>>> labels. Using ‘%=’ (see AssemblerTemplate) may help resolve this 
>>>>>>>> problem.
>>>>>>> Move the assembly blocks that deal with saving and restoring the current
>>>>>>> CPU context into it's own explicitly non-inline functions.  This 
>>>>>>> prevents
>>>>>>> clang from duplicating the assembly blocks.  Just using noinline 
>>>>>>> attribute
>>>>>>> seems to be enough to prevent assembly duplication, in the future 
>>>>>>> noclone
>>>>>>> might also be required if asm block duplication issues arise again.
>>>>>> Wouldn't it be a far easier / less intrusive change to simply append %= 
>>>>>> to
>>>>>> the label names?
>>>>> That won't work AFAICT, as the inline asm in check_wakeup_from_wait()
>>>>> won't be able to make a jump to the .L_wq_resume label defined in the
>>>>> __prepare_to_wait() assembly block if the label is declared as
>>>>> .L_wq_resume%=.
>>>>>
>>>>> Also we want to make sure there's a single .L_wq_resume seeing how
>>>>> check_wakeup_from_wait() uses it as the restore entry point?
>>>> Hmm, yes on both points; the %= would only work for .Lskip. Have you gained
>>>> understanding why there is this duplication? The breaking out of the asm()
>>>> that you do isn't going to be reliable, as in principle the compiler is
>>>> still permitted to duplicate stuff. Afaict the only reliable way is to move
>>>> the code to a separate assembly file (with the asm() merely JMPing there,
>>>> providing a pseudo-return-address by some custom means). Or to a file-scope
>>>> asm(), as those can't be duplicated.
>>>
>>> See the simplified example in
>>> https://github.com/llvm/llvm-project/issues/92161
>>>
>>> When I debugged this a while back, The multiple uses of wqv->esp (one
>>> explicit after the asm, one as an asm parameter) gain pointer
>>> sanitisation, so the structure looks like:
>>>
>>>     ...
>>>     if ( bad pointer )
>>>         __ubsan_report();
>>>     asm volatile (...);
>>>     if ( bad pointer )
>>>         __ubsan_report();
>>>     ...
>>>
>>> which then got transformed to:
>>>
>>>     if ( bad pointer )
>>>     {
>>>         __ubsan_report();
>>>         asm volatile (...);
>>>         __ubsan_report();
>>>     }
>>>     else
>>>         asm volatile (...);
>>
>> But isn't it then going to be enough to latch &wqv->esp into a local 
>> variable,
>> and use that in the asm() and in the subsequent if()?
> 
> I have the following diff which seems to prevent the duplication,
> would you both be OK with this approach?

Yes (with a brief comment added as to the need for the local). And thanks.

Jan

> --- a/xen/common/wait.c
> +++ b/xen/common/wait.c
> @@ -124,6 +124,7 @@ static void __prepare_to_wait(struct waitqueue_vcpu *wqv)
>      struct cpu_info *cpu_info = get_cpu_info();
>      struct vcpu *curr = current;
>      unsigned long dummy;
> +    void *esp = NULL;
>  
>      ASSERT(wqv->esp == NULL);
>  
> @@ -166,12 +167,12 @@ static void __prepare_to_wait(struct waitqueue_vcpu 
> *wqv)
>          ".L_skip:"
>          "pop %%r15; pop %%r14; pop %%r13;"
>          "pop %%r12; pop %%rbp; pop %%rbx"
> -        : "=&S" (wqv->esp), "=&c" (dummy), "=&D" (dummy)
> +        : "=&S" (esp), "=&c" (dummy), "=&D" (dummy)
>          : "0" (0), "1" (cpu_info), "2" (wqv->stack),
>            [sz] "i" (PAGE_SIZE)
>          : "memory", "rax", "rdx", "r8", "r9", "r10", "r11" );
>  
> -    if ( unlikely(wqv->esp == NULL) )
> +    if ( unlikely(esp == NULL) )
>      {
>          gdprintk(XENLOG_ERR, "Stack too large in %s\n", __func__);
>          domain_crash(curr->domain);
> @@ -179,6 +180,7 @@ static void __prepare_to_wait(struct waitqueue_vcpu *wqv)
>          for ( ; ; )
>              do_softirq();
>      }
> +    wqv->esp = esp;
>  }
>  
>  static void __finish_wait(struct waitqueue_vcpu *wqv)
> 




 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.