Open
Description
Tests for builtins are very weak, we should do much better and try to generate random built-in functions to cover various corner cases of the evaluator. Things to do:
- Currently we only have such tests for builtins that take 1 or 2 term arguments. We should at least have tests for 1 and 2 type arguments too, otherwise there’s a real danger of computing the result of a builtin application before costing for it is done, which would be a major bug. Ideally, we should test builtins with interleaved type and term arguments too, to be sure.
- Currently
genTypedTerm
, which we use for testing builtins, always considers type variables to map to Integer. We need to unhardcode that Integer out of there and use arbitrary types in order to improve coverage. - We should consider generating
Arbitrary
BuiltinMeaning
s to test builtins code properly. - Test that no instantiated builtin has a cost that is less than or equal to zero. With the current alignment this also ensures that "no instantiated builtin gives an unlifting failure given arguments that were generated in a type-driven manner" (although that may or may not be true, hard to tell without actually trying it out first), which is also a very good property to test.