Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coverage: Store coverage source regions as Span until codegen #133418

Merged
merged 4 commits into from
Nov 28, 2024

Conversation

Zalathar
Copy link
Contributor

Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.

This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as Span instead.

In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because Span is smaller than 4x u32.


There should be no changes to coverage output.

A used function with no mappings has historically indicated a bug, but that
will no longer be the case after moving some fallible span-processing steps
into codegen.
This will avoid confusion with actual `Span` spans.
@Zalathar Zalathar added the A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) label Nov 24, 2024
@rustbot
Copy link
Collaborator

rustbot commented Nov 24, 2024

r? @estebank

rustbot has assigned @estebank.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Nov 24, 2024
@rustbot
Copy link
Collaborator

rustbot commented Nov 24, 2024

Some changes occurred to MIR optimizations

cc @rust-lang/wg-mir-opt

@jieyouxu
Copy link
Member

jieyouxu commented Nov 24, 2024

Question: is there any kind of rough baseline performance / profile artifact size measurements we can compare this against? I understand if that is uhh non-trivial, but it might be interesting to see if there's any kind of method to track coverage artifact perf/size changes (not for this PR, ofc., just a general remark). I agree this is much nicer maintenance/impl-wise, so merits based on that alone is sufficient for me.

@jieyouxu jieyouxu assigned jieyouxu and unassigned estebank Nov 24, 2024
@Zalathar
Copy link
Contributor Author

Question: is there any kind of rough baseline performance / profile artifact size measurements we can compare this against?

For coverage instrumentation, we currently have very little of that sort of thing. None of the perf benchmarks enable -Cinstrument-coverage, for example.

(Sometimes that's convenient, because it means I don't have to spend time defending coverage-specific regressions caused by necessary refactoring work. But eventually it would be nice to have benchmarks rather than just vibes.)

The main thing we do have is the coverage-map tests, which indicate the size of each function's coverage metadata, and also an estimate of how many physical counters it uses (which affects the size of .profraw output when running the instrumented binary).

But those metrics are unaffected by this PR, which (presumably) just gives a modest decrease in the amount of memory used to store coverage metadata in MIR.

@Zalathar
Copy link
Contributor Author

All that said, for this particular PR any improvement is just gravy, not an intentional goal.

Copy link
Member

@jieyouxu jieyouxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks like a nice robustness improvement to me.

Comment on lines +82 to +88
if try_next && src.as_bytes()[end] == b'{' {
Some(span.with_hi(hi + BytePos(1)))
} else if try_prev && src.as_bytes()[start - 1] == b'}' {
Some(span.with_lo(lo - BytePos(1)))
} else {
None
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remark: :ferrisAware: but this is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm very much aware of why BytePos(1) is bad news in general; in the past I have fixed BytePos(1) bugs in the previous incarnation of this code.

In this very specific set of circumstances, we have confirmed in advance that the code point we're skipping over is a single byte (ASCII { or }), so adjusting by 1 to skip that character really is OK.

@jieyouxu
Copy link
Member

@bors r+ rollup=never (non-trivial coverage handling modifications)

@bors
Copy link
Contributor

bors commented Nov 25, 2024

📌 Commit 2748009 has been approved by jieyouxu

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 25, 2024
@Zalathar
Copy link
Contributor Author

In this case I don't think there's benefit in avoiding rollup:

  • None of the benchmarks enable coverage, so a separate perf run wouldn't do anything useful.
  • This PR only touches coverage-specific code paths and can't realistically break non-coverage tests.

@bors rollup=maybe

joboet added a commit to joboet/rust that referenced this pull request Nov 27, 2024
coverage: Store coverage source regions as `Span` until codegen

Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.

This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as `Span` instead.

In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because `Span` is smaller than 4x u32.

---

There should be no changes to coverage output.
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 27, 2024
Rollup of 6 pull requests

Successful merges:

 - rust-lang#116161 (Stabilize `extended_varargs_abi_support`)
 - rust-lang#132410 (Some more refactorings towards removing driver queries)
 - rust-lang#133418 (coverage: Store coverage source regions as `Span` until codegen)
 - rust-lang#133498 (Add missing code examples on `LocalKey`)
 - rust-lang#133518 (Structurally resolve before checking `!` in HIR typeck)
 - rust-lang#133521 (Structurally resolve before matching on type of projection)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 28, 2024
…iaskrgr

Rollup of 5 pull requests

Successful merges:

 - rust-lang#132410 (Some more refactorings towards removing driver queries)
 - rust-lang#133418 (coverage: Store coverage source regions as `Span` until codegen)
 - rust-lang#133498 (Add missing code examples on `LocalKey`)
 - rust-lang#133518 (Structurally resolve before checking `!` in HIR typeck)
 - rust-lang#133521 (Structurally resolve before matching on type of projection)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit adf9b5f into rust-lang:master Nov 28, 2024
6 checks passed
@rustbot rustbot added this to the 1.85.0 milestone Nov 28, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Nov 28, 2024
Rollup merge of rust-lang#133418 - Zalathar:spans, r=jieyouxu

coverage: Store coverage source regions as `Span` until codegen

Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.

This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as `Span` instead.

In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because `Span` is smaller than 4x u32.

---

There should be no changes to coverage output.
@Zalathar Zalathar deleted the spans branch November 28, 2024 03:47
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 29, 2024
This reverts commit adf9b5f, reversing
changes made to af1ca15.

Reverting due to <rust-lang#133606>.
Zalathar added a commit to Zalathar/rust that referenced this pull request Nov 29, 2024
Revert rust-lang#133418 (Store coverage source regions as `Span`) due to regression rust-lang#133606

This reverts commit adf9b5f, reversing changes made to af1ca15.

Reverting rust-lang#133418 due to regressions reported at rust-lang#133606.

r? jieyouxu
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 29, 2024
Rollup of 7 pull requests

Successful merges:

 - rust-lang#131323 (Support `clobber_abi` in AVR inline assembly)
 - rust-lang#133092 (Always set the deployment target when building std)
 - rust-lang#133134 (Don't use a SyntheticProvider for literally every type)
 - rust-lang#133538 (Better diagnostic for fn items in variadic functions)
 - rust-lang#133590 (Rename `-Zparse-only`)
 - rust-lang#133592 (Misc: better instructions for envrc, ignore `/build` instead of `build/`)
 - rust-lang#133608 (Revert rust-lang#133418 (Store coverage source regions as `Span`) due to regression rust-lang#133606)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Nov 29, 2024
Revert rust-lang#133418 (Store coverage source regions as `Span`) due to regression rust-lang#133606

This reverts commit adf9b5f, reversing changes made to af1ca15.

Reverting rust-lang#133418 due to regressions reported at rust-lang#133606.

r? jieyouxu
bors added a commit to rust-lang-ci/rust that referenced this pull request Dec 11, 2024
coverage: Rearrange the code for embedding per-function coverage metadata

This is a series of refactorings to the code that prepares and embeds per-function coverage metadata records (“covfun records”) in the `__llvm_covfun` linker section of the final binary. The `llvm-cov` tool reads this metadata from the binary when preparing a coverage report.

Beyond general cleanup, a big motivation behind these changes is to pave the way for re-landing an updated version of rust-lang#133418.

---

There should be no change in compiler output, as demonstrated by the absence of (meaningful) changes to coverage tests.

---

try-job: x86_64-gnu
try-job: x86_64-msvc
try-job: aarch64-apple
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 11, 2024
…ouxu

coverage: Rearrange the code for embedding per-function coverage metadata

This is a series of refactorings to the code that prepares and embeds per-function coverage metadata records (“covfun records”) in the `__llvm_covfun` linker section of the final binary. The `llvm-cov` tool reads this metadata from the binary when preparing a coverage report.

Beyond general cleanup, a big motivation behind these changes is to pave the way for re-landing an updated version of rust-lang#133418.

---

There should be no change in compiler output, as demonstrated by the absence of (meaningful) changes to coverage tests.

The first patch is just moving code around, so I suggest looking at the other patches to see the actual changes.

---

try-job: x86_64-gnu
try-job: x86_64-msvc
try-job: aarch64-apple
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Dec 11, 2024
Rollup merge of rust-lang#134163 - Zalathar:covfun, r=SparrowLii,jieyouxu

coverage: Rearrange the code for embedding per-function coverage metadata

This is a series of refactorings to the code that prepares and embeds per-function coverage metadata records (“covfun records”) in the `__llvm_covfun` linker section of the final binary. The `llvm-cov` tool reads this metadata from the binary when preparing a coverage report.

Beyond general cleanup, a big motivation behind these changes is to pave the way for re-landing an updated version of rust-lang#133418.

---

There should be no change in compiler output, as demonstrated by the absence of (meaningful) changes to coverage tests.

The first patch is just moving code around, so I suggest looking at the other patches to see the actual changes.

---

try-job: x86_64-gnu
try-job: x86_64-msvc
try-job: aarch64-apple
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Dec 19, 2024
coverage: Store coverage source regions as `Span` until codegen (take 2)

This is an attempt to re-land rust-lang#133418:

> Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.

> This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as Span instead.

> In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because Span is smaller than 4x u32.

That PR was reverted by rust-lang#133608, because in some circumstances not covered by our test suite we were emitting coverage metadata that was causing `llvm-cov` to exit with an error (rust-lang#133606).

---

The implementation here is *mostly* the same, but adapted for subsequent changes in the relevant code (e.g. rust-lang#134163).

I believe that the changes in rust-lang#134163 should be sufficient to prevent the problem that required the original PR to be reverted. But I haven't been able to reproduce the original breakage in a regression test, and the `llvm-cov` error message is extremely unhelpful, so I can't completely rule out the possibility of this breaking again.

r? jieyouxu (reviewer of the original PR)
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Dec 20, 2024
Rollup merge of rust-lang#134497 - Zalathar:spans, r=jieyouxu

coverage: Store coverage source regions as `Span` until codegen (take 2)

This is an attempt to re-land rust-lang#133418:

> Historically, coverage spans were converted into line/column coordinates during the MIR instrumentation pass.

> This PR moves that conversion step into codegen, so that coverage spans spend most of their time stored as Span instead.

> In addition to being conceptually nicer, this also reduces the size of coverage mappings in MIR, because Span is smaller than 4x u32.

That PR was reverted by rust-lang#133608, because in some circumstances not covered by our test suite we were emitting coverage metadata that was causing `llvm-cov` to exit with an error (rust-lang#133606).

---

The implementation here is *mostly* the same, but adapted for subsequent changes in the relevant code (e.g. rust-lang#134163).

I believe that the changes in rust-lang#134163 should be sufficient to prevent the problem that required the original PR to be reverted. But I haven't been able to reproduce the original breakage in a regression test, and the `llvm-cov` error message is extremely unhelpful, so I can't completely rule out the possibility of this breaking again.

r? jieyouxu (reviewer of the original PR)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-code-coverage Area: Source-based code coverage (-Cinstrument-coverage) S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants