[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [PATCH v5 26/26] xen/riscv: do a 4th linking pass if necessary





On 7/6/26 6:13 PM, Jan Beulich wrote:
On 06.07.2026 17:58, Oleksii Kurochko wrote:
Embedding the symbol table can shift sections and flip relaxation
decisions, changing code size and thus the set of emitted symbols
(e.g. gap end markers).

What difference in symbols is there? (And: Do you mean the ELF symbol
table or what tools/symbols emits?) On ...

The ELF symbol tables of .xen-syms.0 and .xen-syms.1 are identical
(same names, types, and count; only addresses past a certain point
differ). The difference is in what tools/symbols emits: the
address-only "end of symbol" entry produced when want_symbol_end()
is true, i.e. when a sized text symbol's end doesn't reach the next
symbol's address:

    static bool want_symbol_end(unsigned int idx)
    {
        return table[idx].size &&
               (idx + 1 == table_cnt ||
                table[idx].addr + table[idx].size < table[idx + 1].addr);
    }

In pass 1 (generated from .xen-syms.0): simple_strtoull is at
0xffffffffc00c0444 with size 0x16c, ending at 0xffffffffc00c05b0,
while turn_on_mmu (alignment-pinned) starts at 0xffffffffc00c05c0.
16-byte gap -> the end marker at ...c05b0 is emitted.

In pass 2 (generated from .xen-syms.1): simple_strtoull moved to
0xffffffffc00c0454 (same size), ending at exactly 0xffffffffc00c05c0
== turn_on_mmu. No gap -> no end marker. Hence symbols_addresses
shrinks by 8 and symbols_names by 1 (the marker's empty name).


  Re-link with a regenerated table when its
size differs from the previous pass.

This mirrors commit 35de7285d508 ("Arm: do a 4th linking pass if
necessary"), which riscv didn't receive when the underlying
pass-2-vs-pass-3 check was extended to it in commit aa786d6e3467
("non-x86/symbols: check table sizes don't change between linking
passes 2 and 3").

... Arm the problem was with a symbol (dis)appearing from/in the ELF
symbol table. The situation looks to be different here. Sadly ...

I will rephrase then this paragraph to:

This takes the same remedy as commit 35de7285d508 ("Arm: do a 4th
linking pass if necessary"), though the underlying cause differs:
here no symbol (dis)appears from the ELF symbol table; instead the
set of end markers emitted by tools/symbols legitimately changes,
because want_symbol_end() depends on code layout, and layout differs
between passes due to linker relaxation.

Or it could be just dropped.


Signed-off-by: Oleksii Kurochko <oleksii.kurochko@xxxxxxxxx>
---
I faced this issue again in downstream:
   https://gitlab.com/xen-project/people/olkur/xen/-/jobs/15171254706

It was found that the difference between .xen-syms.0 and .xen-syms.1 is in
alignment(?) gap between turn_on_mmu() and the end of simple_strtoull.
Specifically 0xffffffffc00c05b0 is present in pass 1 but not present in
pass 2 (where end of simple_strtoull() is just equal to ffffffffc00c05c0)
because code was shifted:

```
.xen-syms.0:     file format elf64-littleriscv

...

ffffffffc00c0534:       f2e794e3                bne     a5,a4,ffffffffc00c045c 
<simple_strtoull+0x18>
ffffffffc00c0538:       00280813                addi    a6,a6,2
ffffffffc00c053c:       f21ff06f                j       ffffffffc00c045c 
<simple_strtoull+0x18>
ffffffffc00c0540:       00068813                mv      a6,a3
ffffffffc00c0544:       00800613                li      a2,8
ffffffffc00c0548:       f15ff06f                j       ffffffffc00c045c 
<simple_strtoull+0x18>
ffffffffc00c054c:       00277713                andi    a4,a4,2
ffffffffc00c0550:       04070463                beqz    a4,ffffffffc00c0598 
<simple_strtoull+0x154>
ffffffffc00c0554:       fe06869b                addiw   a3,a3,-32
ffffffffc00c0558:       0ff6f693                zext.b  a3,a3
ffffffffc00c055c:       fc96879b                addiw   a5,a3,-55
ffffffffc00c0560:       04c7f263                bgeu    a5,a2,ffffffffc00c05a4 
<simple_strtoull+0x160>
ffffffffc00c0564:       02a60533                mul     a0,a2,a0
ffffffffc00c0568:       00f50533                add     a0,a0,a5
ffffffffc00c056c:       00180813                addi    a6,a6,1
ffffffffc00c0570:       00084683                lbu     a3,0(a6)
ffffffffc00c0574:       0006879b                sext.w  a5,a3
ffffffffc00c0578:       00d30733                add     a4,t1,a3
ffffffffc00c057c:       00074703                lbu     a4,0(a4)
ffffffffc00c0580:       04477893                andi    a7,a4,68
ffffffffc00c0584:       02088063                beqz    a7,ffffffffc00c05a4 
<simple_strtoull+0x160>
ffffffffc00c0588:       00477893                andi    a7,a4,4
ffffffffc00c058c:       fc0880e3                beqz    a7,ffffffffc00c054c 
<simple_strtoull+0x108>
ffffffffc00c0590:       fd07879b                addiw   a5,a5,-48
ffffffffc00c0594:       fcdff06f                j       ffffffffc00c0560 
<simple_strtoull+0x11c>
ffffffffc00c0598:       fc97879b                addiw   a5,a5,-55
ffffffffc00c059c:       fc5ff06f                j       ffffffffc00c0560 
<simple_strtoull+0x11c>
ffffffffc00c05a0:       00000513                li      a0,0
ffffffffc00c05a4:       00058463                beqz    a1,ffffffffc00c05ac 
<simple_strtoull+0x168>
ffffffffc00c05a8:       0105b023                sd      a6,0(a1)
ffffffffc00c05ac:       00008067                ret
         ...

ffffffffc00c05c0 <turn_on_mmu>:
ffffffffc00c05c0:       12000073                sfence.vma
ffffffffc00c05c4:       00800293                li      t0,8
ffffffffc00c05c8:       03c29293                slli    t0,t0,0x3c
ffffffffc00c05cc:       000fc317                auipc   t1,0xfc
ffffffffc00c05d0:       a3430313                addi    t1,t1,-1484 # 
ffffffffc01bc000 <stage1_pgtbl_root>
ffffffffc00c05d4:       00c35313                srli    t1,t1,0xc
ffffffffc00c05d8:       00536333                or      t1,t1,t0
ffffffffc00c05dc:       18031073                csrw    satp,t1
ffffffffc00c05e0:       00050067                jr      a0

ffffffffc00c05e4 <_ident_end>:
ffffffffc00c05e4:       0000                    .insn   2, 0x0000
         ...
```

... with all of this disassembly provided it still doesn't become clear what
changed from .xen-syms.0 to .xen-syms.1.

The disassembly was meant to show the 16-byte padding gap between the
end of simple_strtoull() and (aligned) turn_on_mmu() in .xen-syms.0.
In .xen-syms.1 that gap is gone, which is what removes the end marker
and shrinks symbols_addresses by 8 and symbols_names by 1, as
explained above.

The actual code change is earlier in .text. Inserting the symbol
table (~24k of .rodata) pushes .init.text from 0xffffffffc0148000
to 0xffffffffc014e000. Several calls from .text into .init.text
(e.g. to alloc_boot_pages()/init_boot_pages()) thereby cross JAL's
±1MB reach, so the linker can no longer relax them:

.xen-syms.0:
    <caller>: jal   ffffffffc0152000 <alloc_boot_pages>

.xen-syms.1:
    <caller>: auipc ra,0xfc
              jalr  1044(ra) # ffffffffc015800c <alloc_boot_pages>

Four such call sites grow .text by 16 bytes in total, which is what
shifts simple_strtoull by +0x10 and closes the padding gap before
the (aligned) turn_on_mmu.

~ Oleksii





 


Rackspace

Lists.xenproject.org is hosted with RackSpace, monitoring our
servers 24x7x365 and backed by RackSpace's Fanatical Support®.