Recursively call setup_suites in subdirectories #979
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thank you for maintaining BATS. :)
I modified
bats-exec-suite
so as to execute {setup,teardown}_suite on every subdirectory on a recursive run. The basic idea is to replace the file execution loop (callingbats-exec-file
) with a tail recursion and to enter a subshell whenever the next test is in the subdirectory. In the subshell, it callssetup_suite
of that subdirectory (if it exists), executes the test, and callsteardown_suite
before changing the directory.The motivation was that, if somebody (like, me) tries to organize their tests using the directory structure, the tests in the subdirectory probably need to assume the test environment of their parent, PLUS their own environment if needed. This is doable if BATS recursively recognizes {setup,teardown}_suites in the subdirectories.
That said, there are some concerns even from my side. I think these need to be addressed before this PR is eventually merged into the master.
This is the most obvious concern. I totally agree with BATS's basic policy of being lightweight. I ensured this modification causes no additional overhead when it's not recursive, but I don't know whether a tail recursive is sizably heavier than a loop. For reference, the
time
difference between the unmodified and modified ones was negligible using the test directory structure intest/fixture/suite/recursive_setup
.setup_suite
I think this can be a little misnomer when this recursive setup takes effect. A 'suite' arguably means the set of tests, namely all the tests that BATS will execute. From this perspective, 'setup_suite' should be called only once before the entire test begins. However, I need to clarify that the motivation signified something "between" setup_file and setup_suite at a directory level. Maybe there can be several solutions: sticking to
setup_suite
for backward compatibility, newly creating the concept ofsetup_dir
, or aggressively changing the namesetup_suite
tosetup_dir
.My implementation doesn't support parallel execution. There's nothing impossible to have this "feature" in parallel execution, but it'll definitely need some engineering effort.
I also need to make it clear that I ran all tests in
test
(i.e.,test.bats
andsuite.bats
) and confirmed that they all passed. I also added a new test for this PR in thetest
directory.Thank you. :)