Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Use weak linkage instead of compiler generated shims #134522

Draft
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

bjorn3
Copy link
Member

@bjorn3 bjorn3 commented Dec 19, 2024

#86844 (unstably) made it possible to avoid the allocator shim when using #[global_allocator]. This PR makes it possible to also avoid the allocator shim when using the default allocator in libstd by making use of weak linkage. Eventual stabilization of avoiding the allocator shim may be blocked on it working with the default allocator too.

This is still keeping the __rust_no_alloc_shim_is_unstable symbol for now until #123015 gets stabilized.

TODO: Update comments everywhere, test on macOS and Windows and write a better PR description why we want this. Also prevent codegen of weak symbols when there is a non-weak definition of the same symbol in the same codegen unit.

try-zzz-job: x86_64-msvc
try-job: x86_64-apple-1
try-job: x86_64-apple-2

@rustbot
Copy link
Collaborator

rustbot commented Dec 19, 2024

r? @fee1-dead

rustbot has assigned @fee1-dead.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Dec 19, 2024
@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the weak_linkage_allocator branch from f4f6936 to 93bd5d3 Compare December 19, 2024 16:38
@rust-log-analyzer

This comment has been minimized.

@bjorn3 bjorn3 force-pushed the weak_linkage_allocator branch 5 times, most recently from d179738 to 1b00966 Compare December 20, 2024 09:28
@rust-log-analyzer

This comment has been minimized.

This is still keeping the allocator shim for the oom handler and the
__rust_no_alloc_shim_is_unstable symbol for now.

TODO: Update comments everywhere and test on macOS and Windows
@bjorn3 bjorn3 force-pushed the weak_linkage_allocator branch from 8f98f5e to 08bddfb Compare December 20, 2024 10:53
@bjorn3
Copy link
Member Author

bjorn3 commented Dec 20, 2024

@bors try

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 20, 2024
[WIP] Use weak linkage instead of compiler generated shims

rust-lang#86844 (unstably) made it possible to avoid the allocator shim when using `#[global_allocator]`. This PR makes it possible to also avoid the allocator shim when using the default allocator in libstd by making use of weak linkage. Eventual stabilization of avoiding the allocator shim may be blocked on it working with the default allocator too.

This is still keeping the `__rust_no_alloc_shim_is_unstable` symbol for now until rust-lang#123015 gets stabilized.

TODO: Update comments everywhere, test on macOS and Windows and write a better PR description why we want this. Also prevent codegen of weak symbols when there is a non-weak definition of the same symbol in the same codegen unit.

try-job: x86_64-msvc
try-job: x86_64-apple-1
try-job: x86_64-apple-2
@bors
Copy link
Contributor

bors commented Dec 20, 2024

⌛ Trying commit abd406c with merge 25f4036...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Dec 20, 2024

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 20, 2024
@rust-log-analyzer

This comment has been minimized.

@bjorn3
Copy link
Member Author

bjorn3 commented Dec 20, 2024

@bors try

@bors
Copy link
Contributor

bors commented Dec 20, 2024

⌛ Trying commit 3e25d5a with merge 3659c42...

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 20, 2024
[WIP] Use weak linkage instead of compiler generated shims

rust-lang#86844 (unstably) made it possible to avoid the allocator shim when using `#[global_allocator]`. This PR makes it possible to also avoid the allocator shim when using the default allocator in libstd by making use of weak linkage. Eventual stabilization of avoiding the allocator shim may be blocked on it working with the default allocator too.

This is still keeping the `__rust_no_alloc_shim_is_unstable` symbol for now until rust-lang#123015 gets stabilized.

TODO: Update comments everywhere, test on macOS and Windows and write a better PR description why we want this. Also prevent codegen of weak symbols when there is a non-weak definition of the same symbol in the same codegen unit.

try-job: x86_64-msvc
try-job: x86_64-apple-1
try-job: x86_64-apple-2
@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Dec 20, 2024

💔 Test failed - checks-actions

@bjorn3
Copy link
Member Author

bjorn3 commented Dec 20, 2024

@bors try

@bors
Copy link
Contributor

bors commented Dec 20, 2024

⌛ Trying commit 3e25d5a with merge f0b0345...

bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 20, 2024
[WIP] Use weak linkage instead of compiler generated shims

rust-lang#86844 (unstably) made it possible to avoid the allocator shim when using `#[global_allocator]`. This PR makes it possible to also avoid the allocator shim when using the default allocator in libstd by making use of weak linkage. Eventual stabilization of avoiding the allocator shim may be blocked on it working with the default allocator too.

This is still keeping the `__rust_no_alloc_shim_is_unstable` symbol for now until rust-lang#123015 gets stabilized.

TODO: Update comments everywhere, test on macOS and Windows and write a better PR description why we want this. Also prevent codegen of weak symbols when there is a non-weak definition of the same symbol in the same codegen unit.

try-zzz-job: x86_64-msvc
try-job: x86_64-apple-1
try-job: x86_64-apple-2
@rust-log-analyzer

This comment has been minimized.

@rustbot rustbot added A-run-make Area: port run-make Makefiles to rmake.rs PG-exploit-mitigations Project group: Exploit mitigations labels Dec 20, 2024
@rust-log-analyzer
Copy link
Collaborator

The job x86_64-apple-1 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
Updating files:  98% (50262/50914)
Updating files:  99% (50405/50914)
Updating files: 100% (50914/50914)
Updating files: 100% (50914/50914), done.
Switched to a new branch 'try'
branch 'try' set up to track 'origin/try'.
[command]/usr/local/bin/git log -1 --format=%H
f0b034562a44d58633af9414efbd4c6416626e83
##[group]Run src/ci/scripts/setup-environment.sh
src/ci/scripts/setup-environment.sh
---
file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config remote.upstream.fetch=+refs/heads/*:refs/remotes/upstream/*
file:.git/config submodule.library/backtrace.active=true
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
---
---- [run-make] tests/run-make/bin-emit-no-symbols stdout ----

error: rmake recipe failed to complete
status: exit status: 101
command: cd "/Users/runner/work/rust/rust/build/x86_64-apple-darwin/test/run-make/bin-emit-no-symbols/rmake_out" && env -u RUSTFLAGS AR="ar" CARGO="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage1-tools-bin/cargo" CC="/Users/runner/work/rust/rust/clang+llvm-15.0.7-x86_64-apple-darwin21.0/bin/clang" CC_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64 --target=x86_64-apple-macosx10.12" CXX="/Users/runner/work/rust/rust/clang+llvm-15.0.7-x86_64-apple-darwin21.0/bin/clang++" CXX_DEFAULT_FLAGS="-ffunction-sections -fdata-sections -fPIC -m64 --target=x86_64-apple-macosx10.12 -stdlib=libc++" DYLD_LIBRARY_PATH="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage0-bootstrap-tools/x86_64-apple-darwin/release/deps:/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage0/lib:/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2-tools-bin:/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib" HOST_RPATH_DIR="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2/lib" LD_LIB_PATH_ENVVAR="DYLD_LIBRARY_PATH" LLVM_BIN_DIR="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/ci-llvm/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegendata codegentypes core coroutines coverage csky cskyasmparser cskycodegen cskydesc cskydisassembler cskyinfo debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerclassic dwarflinkerparallel dwp engine executionengine extensions filecheck frontenddriver frontendhlsl frontendoffloading frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo hipstdpar instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcdebugging orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld sandboxir scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi textapibinaryreader transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/ci-llvm/bin/FileCheck" NODE="/usr/local/bin/node" PYTHON="/usr/bin/python3" RUSTC="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2/bin/rustc" RUSTDOC="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2/bin/rustdoc" SOURCE_ROOT="/Users/runner/work/rust/rust" TARGET="x86_64-apple-darwin" TARGET_RPATH_DIR="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage2/lib/rustlib/x86_64-apple-darwin/lib" TARGET_RPATH_ENV="/Users/runner/work/rust/rust/build/x86_64-apple-darwin/test/run-make/bin-emit-no-symbols/rmake_out:/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage0-bootstrap-tools/x86_64-apple-darwin/release/deps:/Users/runner/work/rust/rust/build/x86_64-apple-darwin/stage0/lib" "/Users/runner/work/rust/rust/build/x86_64-apple-darwin/test/run-make/bin-emit-no-symbols/rmake"
--- stderr -------------------------------
assert_contains:
assert_contains:
=== HAYSTACK ===
File: app.o
File: app.o
Format: Mach-O 64-bit x86-64
Arch: x86_64
AddressSize: 64bit
Symbols [
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked17h0a520882f3b2cc3dE (69)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x0
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked18precondition_check17h16e2b060a144ff12E (367)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x30
  }
  Symbol {
  Symbol {
    Name: l___unnamed_1 (482)
    Type: Section (0xE)
    Section: __const (0x3)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xC4
  }
  Symbol {
    Name: GCC_except_table1 (464)
    Name: GCC_except_table1 (464)
    Type: Section (0xE)
    Section: __gcc_except_tab (0x2)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xB4
  }
  Symbol {
  Symbol {
    Name: __ZN3app3oom17h8cfc2a66c883d686E (278)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x90
  }
  Symbol {
    Name: ___rust_alloc_error_handler (22)
    Name: ___rust_alloc_error_handler (22)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xA0
  }
  Symbol {
    Name: _rust_begin_unwind (50)
    Name: _rust_begin_unwind (50)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x70
  }
  Symbol {
  Symbol {
    Name: _rust_eh_personality (1)
    PrivateExtern
    Type: Section (0xE)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x80
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout19is_size_align_valid17h56e3ffe8dcbf744cE (146)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking14panic_nounwind17h6c494ce30a2dc7f2E (311)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking19panic_cannot_unwind17haff82019bed171a8E (217)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
---

=== NEEDLE ===
rust_begin_unwind
assert_contains:
=== HAYSTACK ===
File: app.o
File: app.o
Format: Mach-O 64-bit x86-64
Arch: x86_64
AddressSize: 64bit
Symbols [
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked17h0a520882f3b2cc3dE (69)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x0
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked18precondition_check17h16e2b060a144ff12E (367)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x30
  }
  Symbol {
  Symbol {
    Name: l___unnamed_1 (482)
    Type: Section (0xE)
    Section: __const (0x3)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xC4
  }
  Symbol {
    Name: GCC_except_table1 (464)
    Name: GCC_except_table1 (464)
    Type: Section (0xE)
    Section: __gcc_except_tab (0x2)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xB4
  }
  Symbol {
  Symbol {
    Name: __ZN3app3oom17h8cfc2a66c883d686E (278)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x90
  }
  Symbol {
    Name: ___rust_alloc_error_handler (22)
    Name: ___rust_alloc_error_handler (22)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xA0
  }
  Symbol {
    Name: _rust_begin_unwind (50)
    Name: _rust_begin_unwind (50)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x70
  }
  Symbol {
  Symbol {
    Name: _rust_eh_personality (1)
    PrivateExtern
    Type: Section (0xE)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x80
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout19is_size_align_valid17h56e3ffe8dcbf744cE (146)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking14panic_nounwind17h6c494ce30a2dc7f2E (311)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking19panic_cannot_unwind17haff82019bed171a8E (217)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
---

=== NEEDLE ===
rust_eh_personality
assert_contains:
=== HAYSTACK ===
File: app.o
File: app.o
Format: Mach-O 64-bit x86-64
Arch: x86_64
AddressSize: 64bit
Symbols [
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked17h0a520882f3b2cc3dE (69)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x0
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout25from_size_align_unchecked18precondition_check17h16e2b060a144ff12E (367)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x30
  }
  Symbol {
  Symbol {
    Name: l___unnamed_1 (482)
    Type: Section (0xE)
    Section: __const (0x3)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xC4
  }
  Symbol {
    Name: GCC_except_table1 (464)
    Name: GCC_except_table1 (464)
    Type: Section (0xE)
    Section: __gcc_except_tab (0x2)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xB4
  }
  Symbol {
  Symbol {
    Name: __ZN3app3oom17h8cfc2a66c883d686E (278)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x90
  }
  Symbol {
    Name: ___rust_alloc_error_handler (22)
    Name: ___rust_alloc_error_handler (22)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0xA0
  }
  Symbol {
    Name: _rust_begin_unwind (50)
    Name: _rust_begin_unwind (50)
    PrivateExtern
    Extern
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x70
  }
  Symbol {
  Symbol {
    Name: _rust_eh_personality (1)
    PrivateExtern
    Type: Section (0xE)
    Type: Section (0xE)
    Section: __text (0x1)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Value: 0x80
  }
  Symbol {
  Symbol {
    Name: __ZN4core5alloc6layout6Layout19is_size_align_valid17h56e3ffe8dcbf744cE (146)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking14panic_nounwind17h6c494ce30a2dc7f2E (311)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)
    Flags [ (0x0)
    ]
    Value: 0x0
  }
  Symbol {
    Name: __ZN4core9panicking19panic_cannot_unwind17haff82019bed171a8E (217)
    Type: Undef (0x0)
    Section:  (0x0)
    RefType: UndefinedNonLazy (0x0)
    Flags [ (0x0)

@bors
Copy link
Contributor

bors commented Dec 21, 2024

☔ The latest upstream changes (presumably #134604) made this pull request unmergeable. Please resolve the merge conflicts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-run-make Area: port run-make Makefiles to rmake.rs PG-exploit-mitigations Project group: Exploit mitigations S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants