Skip to content

Warn when comparing against float.Epsilon/double.Epsilon #7528

Open
@lcsondes

Description

Describe the problem you are trying to solve

Comparing floating-point values often requires a certain amount of tolerance instead of using the == operator to check for equality.

A widely-used anti-pattern is replacing a == b with MathF.Abs(a - b) < float.Epsilon or Math.Abs(a - b) < double.Epsilon.
At a glance, these look like comparisons with a small amount of tolerance, but due to the definition of Epsilon, the only number that can satisfy these checks is 0.0, which makes these == in disguise.

On platforms where denormals are flushed to zero, Epsilon == 0.0, which is even worse.

Describe suggestions on how to achieve the rule

Issue a warning that lives in Microsoft.CodeAnalysis.NetAnalyzers, enabled by default, when a floating-point value is found in a less-than relationship with the corresponding Epsilon constant of its type, such as f < float.Epsilon or double.Epsilon > d, where f and d are arbitrary expressions of type float and double, respectively.

In case the comparand is NOT an Abs(a - b)-like expression, I think that this is still warning-worthy, because x < Epsilon is just an obfuscated way of expressing x <= 0. It is extremely unlikely that this is the intended outcome of such an expression, and not a comparison with actual tolerance.

If this is considered too broad, only warn for Abs(a - b)-like expressions on the other side of the comparison. I have no good definition for this. If the compiler somehow knows that an expression cannot be negative at runtime, this information could be used.

Do NOT issue warnings if a double is compared against float.Epsilon (note the mismatched types), or a <= / >= comparison is used, but consider issuing a different warning, since this is still probably wrong (but not guaranteed wrong).

Additional context

Consider an additional, more generic warning for all comparisons against denormal constants. This additional warning could be limited to projects that actually target affected platforms (or Any CPU), and omitted for, e.g., x64-only projects.

dotnet/runtime could have benefited from such a warning. Every single one of its uses of Epsilon fell into this trap, except the dynamic test, because that one doesn't even call Abs():

float https://github.com/search?q=repo%3Adotnet%2Fruntime+%22%3C+float.epsilon%22
double https://github.com/search?q=repo%3Adotnet%2Fruntime+%22%3C+double.epsilon%22

Related issue: dotnet/runtime#62714

Related, but different suggestions: #4480, dotnet/runtime#28054

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions