compiletest: Deficiencies and pitfalls of //@ normalize-*
headers #126372
Open
Description
These headers allow individual tests to define custom output-normalization rules, so that output snapshots are less sensitive to things like platform differences, line numbers, and timestamps.
Unfortunately, they have a few limitations and pitfalls:
- The value parser for these headers is very lax, and when it does fail to parse, the header is silently ignored.
- The parser doesn't require a colon
:
after the header name, so the test suite contains an inconsistent mix of with-colon and without-colon rules. - The rules cannot contain literal double-quote
"
characters, which is very awkward for some kinds of output (e.g. JSON). - The parser uses
parse_cfg_name_directive
to help read the header name, which is sometimes useful (e.g. for defining separate rules for 32-bit and 64-bit), but most of the time it means that tests have to awkwardly writenormalize-stderr-test
instead of the more intuitivenormalize-stderr
. - The rules are applied to both compile output and run output, which is usually not a problem in practice, but can be unexpected.
- Like many other headers, they are only meaningful in
ui
-mode tests, and are silently ignored if they appear in other kinds of test.
@rustbot label +A-compiletest
Metadata
Assignees
Labels
Type
Projects
Status
Backlog