Ref #17165 (comment), we should try to refactor what's currently in test/runtests.jl to instead be a default part of the Base.Test module so packages can also use it for pretty-printing, parallel execution, running of selective subsets of tests (related to #15404), and everything else that base's tests do but packages have to roll their own.