Open
Description
In https://github.com/web-platform-tests/rfcs/blob/master/rfcs/web_features.md we introduced the ability to associate tests with features using WEB_FEATURES.yml
files, and showing them on wpt.fyi like this:
- https://wpt.fyi/results/?label=master&label=experimental&aligned&q=feature%3Acanvas-text-baselines
- https://wpt.fyi/results/?label=master&label=experimental&aligned&q=feature%3Afont-palette
- https://wpt.fyi/results/?label=master&label=experimental&aligned&q=feature%3Anesting
There are now over 150 features mapped using WEB_FEATURES.yml
files. To allow more people to contribute to this, it's clear that some guidelines will be needed.
Questions and tentative answers:
- When should a test be linked to a feature?
- When it's in the scope of the feature in web-features judging by its description and
compat_features
list. - Tests should match the spec, but contributors aren't expected to verify this, assume the initial review covered this. If it becomes known that a test doesn't match the spec, the best course of action is probably to make it tentative.
- When it's in the scope of the feature in web-features judging by its description and
- Can a test be linked to multiple features?
- Yes! But this is only useful when two features ship in different orders in different browsers, to allow filtering out tests that cover a yet-to-be-implemented feature.
- How can a specific test be excluded?
- There's no syntax for this, but the need is obvious. Right now the only way is to list all tests and remove some.
- What about tentative tests?
- It's fine if they're captured by the patterns, they can be excluded using
!is:tentative
on wpt.fyi if desired.
- It's fine if they're captured by the patterns, they can be excluded using
- What about subtests?
- There's no way to associate only a few subtests with a feature. This hasn't been a big problem in practice, but in a few cases this makes it impossible to capture all the tests.
Beyond this, some form of test suite review by feature experts would be useful, but I'll file a separate issue about that when I've thought through which repo it belongs in.
cc @jcscottiii
Metadata
Assignees
Labels
No labels