Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add coverage statistics #5906

Open
2 tasks done
mildm8nnered opened this issue Dec 22, 2024 · 0 comments · May be fixed by #5907
Open
2 tasks done

Add coverage statistics #5906

mildm8nnered opened this issue Dec 22, 2024 · 0 comments · May be fixed by #5907
Labels
enhancement Ideas for improvements of existing features and rules.

Comments

@mildm8nnered
Copy link
Collaborator

New Issue Checklist

Feature or Enhancement Proposal

It would be nice if SwiftLint could report "coverage" statistics. In the context of a linter, what does coverage mean?

  • If every rule was applied to every line of code, then coverage would be 1, or 100%.
  • If no rules are applied to any lines of code, then coverage would be zero.
  • If half the rules are applied to all of the code, then coverage should be .5 or 50%.
  • If all the rules are applied to half of the code, then coverage should also be .5

So we can define coverage as

sum of the number_of_rules_applied_to_each_line / (total_number_of_lines * number_of_rules)

number_of_rules can be defined in two ways:

The number of enabled rules ("Enabled rules coverage")
The total number of rules, enabled or not ("All rules coverage")

The first metric ("Enabled rules coverage") provides a measure of how much enabled rules are being suppressed with commands like swiftlint:disable. Generally we'd expect this coverage figure to be very high, but if some malicious actor was to go around writing // swiftlint:disable all at the top of every file, it would drop to zero. For SwiftLint itself, coverage on this metric is .995.

The second metric ("All rules coverage") provides a measure what coverage you're getting, compared to the coverage you would get if all rules were enabled. A configuration that enabled only one rule would have very low coverage - as some rules are contradictory, in a real deployment this figure would rarely be 1. For SwiftLint itself, coverage on this metric is .861.

The corresponding figures for Periphery are 1, and 0.36, as not that many rules are enabled in Periphery's configuration.

A few other definitions:

number of lines of code - as SwiftLint can flag up violations in blank lines or comments as well as code, it makes more sense to count every line in every file, rather than lines which actually contain code per se.

@mildm8nnered mildm8nnered linked a pull request Dec 22, 2024 that will close this issue
@SimplyDanny SimplyDanny linked a pull request Dec 22, 2024 that will close this issue
@SimplyDanny SimplyDanny added the enhancement Ideas for improvements of existing features and rules. label Dec 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Ideas for improvements of existing features and rules.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants