What's Changed

Structured Results

We invite users to try out a preview of Structured Results, the main feature from this release candidate. For more details on the feature, please check out the first paragraph of our probes README as well as our blog post.

At a high level, structured results involves breaking the existing 19 Scorecard Checks into individual heuristics so users can pick and choose which ones they care about. You can see a list of all supported probes by checking out the probes/ directory. To run individual probes, use the --probes CLI flag with a comma separated list of names. You must also specify the --format probe option to see the results. Please run scorecard --help if you need more details.

Example:

scorecard --repo github.com/ossf/scorecard --probes archived,fuzzed,hasLicenseFile --format probe

Check Enhancements and Bug Fixes

• Branch-Protection
  • ✨ Branch Protection check now also evaluates if the project requires PRs prior to make changes to the branch. This won't change anything for the users that already require reviews, but will enable score enhancement for those who can't require reviewers. (#3499, @diogoteles08)
• Dependency-Update-Tool
  • ✨ Dependency-Update-Tool now detects Renovate config files in a .gitlab folder. (#3823, @spencerschrock)
  • 🐛 Sonatype Lift is no longer recognized as a Dependency-Update-Tool because it is retired. (#3605, @spencerschrock)
  • 🐛 Dependency-Update-Tool: ignore search commit data for repo clients which dont support it by @spencerschrock in #3756
• Fuzzing
  • ⚠️ Remove OneFuzz from fuzzing checks by @DavidKorczynski in #3666
• Pinned-Dependencies
  • 🐛 Pinned-Dependencies now continues after encountering runtime errors (#3515, @pnacht)
  • 🐛 Scorecard no longer considers unpinned Dockerfiles in vendor and third_party directories. (#3675, @AdamKorcz)
  • 🐛 Files downloaded by Git SHA from GitHub and executed are no longer considered as not pinned by hash. (#3694, @martincostello)
  • 🐛 Shell commands in Dockerfile here-documents are now parsed correctly by the Pinned-Dependencies check (#3774, @jkreileder)
• Signed-Releases
  • 🐛 Fixed a bug which allowed some repos to score higher than 10 in the Signed-Releases check. (#3768, @spencerschrock)
  • ✨ Support .sigstore bundles to check for signed releases (#3772, @edgarrmondragon)
• Vulnerabilities
  • 🐛 Projects without dependencies or packages no longer throw an error for the Vulnerabilities check. (#3803, @spencerschrock)
  • 🐛 Go stdlib vulns are removed Vulnerabilities check output (#3925, @spencerschrock)

RepoClient Improvements

• GitHub

  • 🐛 Scorecard processes commit activity from large GitHub repos in chunks to avoid timeout issues (#3680, @spencerschrock)

• GitLab

  • 🐛 Fix scanning for GitLab private repositories. (#3596, @gabibguti)
  • ✨ Added --commit-depth support for GitLab repos (#3672, @ashearin)
  • 🐛 Parse Gitlab Status fields to align w/Github Status and Conclusion by @ashearin in #3706
  • 🐛 Fix signed release error for empty gitlab repo by @naveensrinivasan in #3753
  • 🐛 Scorecard no longer crashes on GitLab repos with no commits (#3731, @ashearin)
  • 🐛 Fixed a bug which prevented Scorecard from analyzing some self-hosted GitLab repos. (#3819, @spencerschrock)

• Local Directory

  • 🐛 ignore .git folder for localdir by @naveensrinivasan in #3943

Other

• 🐛 Fix nils by @naveensrinivasan in #3750
• ✨ Added logic to ensure check scores are between 0 and 10 (#3769, @spencerschrock)

Breaking Changes

• File access through RepoClient now returns an io.ReadCloser, instead of the full file contents. (#3912, @spencerschrock). This enabled fixing two bugs which affect very large repos.
  • 🐛 Limit Binary Artifact file reads to first 1024 bytes by @spencerschrock in #3923
  • 🐛 Avoid reading every file searching for sonar configs by @spencerschrock in #3929
• ⚠️ refactor: rename fields on Branch Protection Pull Request rules by @diogoteles08 in #3879
• ⚠️ removerule.Remediation and switch users to probe.Remediation by @spencerschrock in #3978

Docs

• 📖 fix typo by @AdamKorcz in #3699
• 📖 Added beginner's guide to scorecard checks docs by @ariathaker in #3617
• 📖 fixup transposition typos in remediation package copy by @daveworth in #3734
• 📖 Update README with zoom meeting info by @leec94 in #3739
• 📖 Clarify lack of 2FA check in README by @raghavkaul in #3784
• 📖 Add documentation about probes and contributing by @AdamKorcz in #3762
• 📖 Spelling by @jsoref in #3804
• 📖 Update contributor ladder to reduce duration requirements by @afmarcum in #3899
• 📖 Update slack image by @afmarcum in #3906
• 📖 Document that .sigstore bundles are part of check for Signed-Releases (#3922, @cpswan)
• 📖 Add survey announcement to readme by @afmarcum in #3942
• 📖 Review and update CONTRIBUTING.md by @spencerschrock in #4002
• 📖 revert PAT scope change and document Go resources by @spencerschrock in #4003

New Contributors

• @ashearin made their first contribution in #3672
• @ariathaker made their first contribution in #3617
• @daveworth made their first contribution in #3734
• @edgarrmondragon made their first contribution in #3772
• @manishtiwari25 made their first contribution in #3732
• @jkreileder made their first contribution in #3774
• @tuminoid made their first contribution in #3783
• @lelia made their first contribution in #3822
• @jsoref made their first contribution in #3804
• @jitsengupta17 made their first contribution in #3302
• @cpswan made their first contribution in #3922
• @adamdmharvey made their first contribution in #3972
• @fhoeborn made their first contribution in #3838

Full Changelog: v4.13.1...v5.0.0-rc1