Stabilize -Zgcc-ld=lld
as -Clink-self-contained=linker -Clinker-flavor=gcc-lld
#510
Description
Proposal
In the context of improvements to compile-times, we'd like to make progress on enabling LLD by default. Since this is a broad topic, with known issues and multiple stakeholders, the goal is to first focus on an achievable subset -- enabling LLD by default on linux to start -- and to make incremental progress towards that in multiple steps.
This MCP is proposing such a first step, stabilizing the -Zgcc-ld=lld
flag to be able to use rust-lld
(avoiding the known lld
issues for now, allowing to focus on a smaller scope and make progress), and then other follow-up tasks towards achieving the goal.
The system's lld
can already be used on stable, and we build and distribute rust-lld
in rustup
. Using rust-lld
would alleviate the need to install lld
, and possibly avoid some incompatibilities between the system version and the LLVM version used by rustc (or the need to keep them in sync). On some targets the wrappers and executable can be used directly, but it's not the case by default on linux, and that's where -Zgcc-ld=lld
currently helps.
-Zgcc-ld=lld
uses lld-wrapper
s to call rust-lld
. They are compiled as ld
and ld64
binaries, and distributed in a gcc-ld
folder in the sysroot next to rust-lld
. The flags makes sure the wrappers exist, and adds arguments to the Command
used to do the linking.
Stabilizing -Zgcc-ld=lld
as -Clink-self-contained=linker -Clinker-flavor=gcc-lld
The details of this proposal were discussed in this zulip thread and the discussion there has settled to stabilize a way to use rust-lld
by splitting the 2 things that -Zgcc-ld=lld
does more cleanly:
- it requests using
lld
. We propose that this is done via a dedicated-C linker-flavor=gcc-lld
option that would itself only add the-fuse-ld=lld
link argument. (Or as a colon-separatedgcc:lld
, allowing for extensions such as passing anything after the:
separator straight through-fuse-ld=
and work out of the box forgold
andmold
if supported by the installed version of GCC) - it uses
rust-lld
instead, by passing the path needed to find thelld-wrappers
within the sysroot. We propose this is done via a new option to an existing flag, a-Clink-self-contained=linker
option. This flags currently only targets linking our CRT objects on a few targets.
Currently, the same enums are used internally to handle the CLI's codegen flags, by the target specs themselves, as well as all linking related code. This can cause issues of duplication, and be error-prone: it seems some lld
flavor variants have been created for CLI use, but they must be handled the same way internally. We propose splitting the surface enums used for CLI, so that the changes to the flag values don't change the internal linking code or target specs.
Testing
Looking at the issues in the rust repo, I couldn't find some related to -Zgcc-ld=lld
itself. There are a few about rust-lld
, but on other targets.
I have tested enabling it on x86_64-unknown-linux-gnu
with the 800 most popular crates on crates.io (most of them are libraries so linking and executing only happens for build scripts and proc macros) and a dozen popular binaries (cargo, ripgrep, nushell, tokei, etc) without obvious problems. It's possible there are still issues, in addition to the known issues about using lld
in general: it's unclear whether this flag is well-known and used in the community, so a build-and-test
crater run would be at least reassuring. I've opened PR 96025 to do a crater run with -Zgcc-ld=lld
enabled.
Of note
- distro builds don't bundle
rust-lld
, so-Clink-self-contained=linker
should either be a no-op there, or produce a warning. This could be requested with aconfig.toml
flag, either a new one or the existingrust.lld
flag. - to allow for some experimentation and testing on nightly, the new flag values will be requiring
-Z unstable-options
in the beginning.
Follow-up tasks
These follow-up tasks could be next steps towards using rust-lld
as the default linker on linux:
- Tracking known issues when using
lld
, finding ways to fix them or have workarounds.
There's an issue with stack traces generated when using perf
, detected in flamegraph-rs
. This one doesn't seem to be tracked in the rust or LLVM repositories. The impact in practice is still a bit unclear (if it's not limited to the flamegraph use-case), and it doesn't seem to affect e.g. cachegrind
. It probably should at least be tracked in our issues, since it could be decided to be a blocker. It seems unlikely, as there is a workaround (the --no-rosegment
link arg) and we could imagine using it by default when rust-lld
is enabled.
There is one issue related to coverage on the musl target, but this could be avoided and fixed later, by focusing on enabling it only for x86_64-unknown-linux-gnu
. This specific use-case would then use the existing default. (The workaround for the previous issue doesn't seem work in this case)
Having LLVM/LLD experts look at these would be good. They could know whether these are issues in lld
or our use, their severity, etc.
-
Likely publicize the new flag, so that users can try it out and report issues.
-
Eventually, depending on the results of the previous two tasks, discussing switching the default to
rust-lld
.
Mentors or Reviewers
Maybe @petrochenkov, since they've reviewed the PR adding -Zgcc-ld
?
Process
The main points of the Major Change Process are as follows:
- File an issue describing the proposal.
- A compiler team member or contributor who is knowledgeable in the area can second by writing
@rustbot second
.- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
-C flag
, then full team check-off is required. - Compiler team members can initiate a check-off via
@rfcbot fcp merge
on either the MCP or the PR.
- Finding a "second" suffices for internal changes. If however, you are proposing a new public-facing feature, such as a
- Once an MCP is seconded, the Final Comment Period begins. If no objections are raised after 10 days, the MCP is considered approved.
You can read more about Major Change Proposals on forge.
Comments
This issue is not meant to be used for technical discussion. There is a Zulip stream for that. Use this issue to leave procedural comments, such as volunteering to review, indicating that you second the proposal (or third, etc), or raising a concern that you would like to be addressed.
Activity