Skip to content

[Epic] Generate Arbitrary BuiltinMeanings #6384

Open
@effectfully

Description

Tests for builtins are very weak, we should do much better and try to generate random built-in functions to cover various corner cases of the evaluator. Things to do:

  • Currently we only have such tests for builtins that take 1 or 2 term arguments. We should at least have tests for 1 and 2 type arguments too, otherwise there’s a real danger of computing the result of a builtin application before costing for it is done, which would be a major bug. Ideally, we should test builtins with interleaved type and term arguments too, to be sure.
  • Currently genTypedTerm, which we use for testing builtins, always considers type variables to map to Integer. We need to unhardcode that Integer out of there and use arbitrary types in order to improve coverage.
  • We should consider generating Arbitrary BuiltinMeanings to test builtins code properly.
  • Test that no instantiated builtin has a cost that is less than or equal to zero. With the current alignment this also ensures that "no instantiated builtin gives an unlifting failure given arguments that were generated in a type-driven manner" (although that may or may not be true, hard to tell without actually trying it out first), which is also a very good property to test.

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions