Number parsing rejecting doubles in scientific notation #531
Description
1. Description
Hello, I ran into an issue with parsing doubles using scientific notation. I believe I have tracked the issue down, and I am hoping to provide a fix.
Given a class defining a double
field:
public class Test
{
public double Value { get; set; }
}
The following Dynamic Linq expression results in a ParseException
when passed to DynamicExpressionParser.ParseLambda
:
Value < 1.2345E-4
where this documentation states that 1.2345E-4
is a valid double literal.
This appears to happen because ExpressionParser.TryParseAsDouble
always passes NumberStyles.Number
to double.Parse
here. This flag specifically disallows exponents in doubles: https://docs.microsoft.com/en-us/dotnet/api/system.globalization.numberstyles?view=net-5.0
I noticed that TryParseAsFloat
uses NumberStyles.Float
instead, which does allow exponents. However, parsing as float always fails in my example above, because the expression doesn't add a qualifier which forces it to be recognized as a float.
2. Exception
Here is the exception that is thrown by ParseLambda
:
System.Linq.Dynamic.Core.Exceptions.ParseException
HResult=0x80131500
Message=Invalid real literal '2e2'
Source=System.Linq.Dynamic.Core
StackTrace:
at System.Linq.Dynamic.Core.Parser.ExpressionParser.TryParseAsDouble(String text, Char qualifier)
at System.Linq.Dynamic.Core.Parser.ExpressionParser.TryParseAsDecimal(String text, Char qualifier)
at System.Linq.Dynamic.Core.Parser.ExpressionParser.TryParseAsFloat(String text, Char qualifier)
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseRealLiteral()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParsePrimaryStart()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParsePrimary()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseUnary()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseMultiplicative()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseAdditive()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseShiftOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseComparisonOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseLogicalAndOrOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseIn()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseAndOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseOrOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseLambdaOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseNullCoalescingOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.ParseConditionalOperator()
at System.Linq.Dynamic.Core.Parser.ExpressionParser.Parse(Type resultType, Boolean createParameterCtor)
at System.Linq.Dynamic.Core.DynamicExpressionParser.ParseLambda(Type delegateType, ParsingConfig parsingConfig, Boolean createParameterCtor, ParameterExpression[] parameters, Type resultType, String expression, Object[] values)
at System.Linq.Dynamic.Core.DynamicExpressionParser.ParseLambda(ParsingConfig parsingConfig, Boolean createParameterCtor, ParameterExpression[] parameters, Type resultType, String expression, Object[] values)
at System.Linq.Dynamic.Core.DynamicExpressionParser.ParseLambda[T,TResult](ParsingConfig parsingConfig, Boolean createParameterCtor, String expression, Object[] values)
at ReproduceDynamicLinq.Program.Main() in C:\Users\alweaver\source\repos\ReproduceDynamicLinq\ReproduceDynamicLinq\Program.cs:line 12
3. Fiddle or Project
Fiddle: https://dotnetfiddle.net/trfQjl
Note: There are issues when running this fiddle in-browser, due to incompatibilities between the security attributes on ParseException
and the fact that dotnet fiddle executes code as a Medium Trust application. It ends up throwing a runtime error instead. Running the same code locally reproduces the expected exception.
4. Any further technical details
I am happy to provide a PR to improve the numeric parsing, given a bit of guidance:
- Should we always hardcode the
NumberStyles
enum, or should I go ahead and add the enum toParsingConfig
so that users may adjust it according to their needs?NumberParseCulture
is already configurable in this way, so it seems like the supported number style might also be a good candidate to be a configuration option. I'm happy to make this configurable if we want. - Should the default
NumberStyles
value beNumberStyles.Number | NumberStyles.Exponent
, orNumberStyles.Float
? Technically,NumberStyles.Float
disablesAllowTrailingSign
, whichNumber
currently enables. I presume the first is preferable, to avoid a regression. - Another approach might be to keep the default of
NumberStyles.Number
and only make the flag configurable throughParsingConfig
. This would allow me to set this flag in my ownParsingConfig
and use exponents freely, but it also avoids changing existing behavior of the parser. The drawback here is that the default parse behavior still doesn't match the documentation.
Thanks all :)