Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add tests for linters #4993

Open
4 tasks
BenHenning opened this issue May 23, 2023 · 0 comments
Open
4 tasks

Add tests for linters #4993

BenHenning opened this issue May 23, 2023 · 0 comments
Labels
enhancement End user-perceivable enhancements. Impact: Low Low perceived user impact (e.g. edge cases). Work: High It's not clear what the solution is. Z-ibt Temporary label for Ben to keep track of issues he's triaged.

Comments

@BenHenning
Copy link
Member

Is your feature request related to a problem? Please describe.
#4983 introduced linter wrappers for multiple languages (to replace the existing shell scripts). These require unit tests for behavior regression verification.

Describe the solution you'd like
Each of the new, untested classes in the PR need tests to cover their main behaviors. Below are the new classes and minimum tests to add (more are likely needed, these are just a first pass):

  • Add tests for Buf (suggest using the command locally to get an idea on the sort of output that the class will encounter)
    1. testMain_noArguments_failsWithError
    2. testMain_twoArguments_failsWithError
    3. testMain_nonExistentRepoRoot_failsWithError
    4. testMain_existingRepoRoot_noProtoBufs_passes
    5. testMain_existingRepoRoot_withProtoBufs_noLintErrors_passes
    6. testMain_existingRepoRoot_withProtoBufs_withLintErrors_failsWithError
    7. testRunBuf_nonExistentRepoRoot_throwsException
    8. testRunBuf_noProtoBufs_outputsPass
    9. testRunBuf_withProtoBufs_outsideSupportedDirs_noLintErrors_outputsPass
    10. testRunBuf_withProtoBufs_outsideSupportedDirs_withLintErrors_outputsPass
    11. testRunBuf_withProtoBufs_withinSupportedDirs_noLintErrors_outputsPass
    12. testRunBuf_withProtoBufs_withinSupportedDirs_withLintErrors_throwsException
  • Add tests for Buildifier (suggest using the command locally to get an idea on the sort of output that the class will encounter)
    1. testMain_noArguments_failsWithError
    2. testMain_oneArgument_failsWithError
    3. testMain_twoArguments_invalidMode_failsWithError
    4. testMain_threeArguments_failsWithError
    5. testMain_nonExistentRepoRoot_failsWithError
    6. testMain_check_existingRepoRoot_noBazelFiles_passes
    7. testMain_check_existingRepoRoot_withBazelFiles_noLintErrors_passes
    8. testMain_check_existingRepoRoot_withBazelFiles_withLintErrors_failsWithError
    9. testMain_fix_existingRepoRoot_withBazelFiles_noLintErrors_passes
    10. testMain_fix_existingRepoRoot_withBazelFiles_withLintErrors_passesAndFixesErrors
    11. testRunBuildifier_check_nonExistentRepoRoot_throwsException
    12. testRunBuildifier_check_noBazelFiles_outputsPass
    13. testRunBuildifier_check_withBazelFiles_noLintErrors_outputsPass
    14. testRunBuildifier_check_withBazelFiles_withLintErrors_throwsException
    15. testRunBuildifier_fix_nonExistentRepoRoot_throwsException
    16. testRunBuildifier_fix_noBazelFiles_outputsPass
    17. testRunBuildifier_fix_withBazelFiles_noLintErrors_outputsPassNothingToFix
    18. testRunBuildifier_fix_withBazelFiles_withFixableLintErrors_ouputsPassAndFixesFiles
    19. testRunBuildifier_fix_withBazelFiles_withUnfixableLintErrors_throwsExceptionAndFixesSomeIssues
  • Add tests for Checkstyle (suggest using the command locally to get an idea on the sort of output that the class will encounter)
    1. testMain_noArguments_failsWithError
    2. testMain_twoArguments_failsWithError
    3. testMain_nonExistentRepoRoot_failsWithError
    4. testMain_existingRepoRoot_noJavaFiles_passes
    5. testMain_existingRepoRoot_withJavaFiles_noLintErrors_passes
    6. testMain_existingRepoRoot_withJavaFiles_withLintErrors_failsWithError
    7. testRunCheckstyle_nonExistentRepoRoot_throwsException
    8. testRunCheckstyle_noJavaFiles_outputsPass
    9. testRunCheckstyle_withJavaFiles_outsideSupportedDirs_noLintErrors_outputsPass
    10. testRunCheckstyle_withJavaFiles_outsideSupportedDirs_withLintErrors_outputsPass
    11. testRunCheckstyle_withJavaFiles_withinSupportedDirs_noLintErrors_outputsPass
    12. testRunCheckstyle_withJavaFiles_withinSupportedDirs_withLintErrors_throwsException
  • Add tests for Buildifier (suggest using the command locally to get an idea on the sort of output that the class will encounter)
    1. testMain_noArguments_failsWithError
    2. testMain_oneArgument_failsWithError
    3. testMain_twoArguments_invalidMode_failsWithError
    4. testMain_fiveArguments_failsWithError
    5. testMain_nonExistentRepoRoot_failsWithError
    6. testMain_check_existingRepoRoot_noKotlinFiles_passes
    7. testMain_check_existingRepoRoot_withKotlinFiles_noLintErrors_passes
    8. testMain_check_existingRepoRoot_withKotlinFiles_withLintErrors_failsWithError
    9. testMain_fix_existingRepoRoot_withKotlinFiles_noLintErrors_passes
    10. testMain_fix_existingRepoRoot_withKotlinFiles_withLintErrors_passesAndFixesErrors
    11. testRunKtlint_check_nonExistentRepoRoot_throwsException
    12. testRunKtlint_check_noKotlinFiles_outputsPass
    13. testRunKtlint_check_withKotlinFiles_noLintErrors_outputsPass
    14. testRunKtlint_check_withKotlinFiles_withLintErrors_throwsException
    15. testRunKtlint_fix_nonExistentRepoRoot_throwsException
    16. testRunKtlint_fix_noKotlinFiles_outputsPass
    17. testRunKtlint_fix_withKotlinFiles_noLintErrors_outputsPassNothingToFix
    18. testRunKtlint_fix_withKotlinFiles_withFixableLintErrors_ouputsPassAndFixesFiles
    19. testRunKtlint_fix_withKotlinFiles_withUnfixableLintErrors_throwsExceptionAndFixesSomeIssues
    20. testRunKtlint_generate_noInputScript_throwsException
    21. testRunKtlint_generate_noOutputScript_throwsException
    22. testRunKtlint_generate_nonExistentInputScript_throwsException
    23. testRunKtlint_generate_invalidShellScript_outputsEmptyFile
    24. testRunKtlint_generate_validShellScript_extractsJarFromFile

Describe alternatives you've considered
N/A -- All utilities require tests.

Additional context
Each of the utilities above already have existing placeholder test suites that are ready for new tests to be added, and should already build, run, and pass.

All other script & script utility tests are good to reference when building these. Note that these tests can only be run with Bazel and syntax highlighting/auto completion won't work in Android Studio unless using the Bazel plugin.

Note also that these tests often require creating real test data (such as APKs) to verify their functionality. This should be done without needing to permanently check in any binary or test files to the repository (prefer generating the files at runtime during the test run, instead), aside from changing the existing test suite files.

@MohitGupta121 MohitGupta121 added enhancement End user-perceivable enhancements. Impact: Low Low perceived user impact (e.g. edge cases). Z-ibt Temporary label for Ben to keep track of issues he's triaged. Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet. Work: High It's not clear what the solution is. and removed Work: Medium The means to find the solution is clear, but it isn't at good-first-issue level yet. labels Jun 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement End user-perceivable enhancements. Impact: Low Low perceived user impact (e.g. edge cases). Work: High It's not clear what the solution is. Z-ibt Temporary label for Ben to keep track of issues he's triaged.
Development

No branches or pull requests

3 participants