Skip to content

Unexpected behavior of value-converted expressions in queries #35515

Open
@ranma42

Description

Bug description

Using value converters seems to be very error prone, as they can be freely used within LINQ expressions, but in several cases the resulting SQL query does not take value conversion into account.

Apparently (under the assumption that the value conversion is a pure function and that the two directions are indeed inverse), equality comparison between a value-converted column against a constant or parameter matches the "obvious" behavior (the one you would get through client-side evaluation).

Most other expressions seem to be prone to mismatching translations, which can lead to exceptions and/or silently return "wrong" results.

This is likely related to

IIUC the plan was to add a warning (see the comment #11347 (comment)) but I was unable to trigger it when running the code and I did not it in the documentation.

If there is still a desire to work towards emitting warnings in these cases, this code example might be a good starting point for further testing/bug hunting (I did not find a PR or an issue with this specific goal).

NOTE: I am not sure whether the code I have written is valid or not 😅
I might be misunderstanding the explanation of the limitations of value converters. The closest one I see is "It isn't possible to query into value-converted properties, e.g. reference members on the value-converted .NET type in your LINQ queries.", but it seems to be aimed specifically at member access, while the issue I am hitting involves simple scalar values (or, rather, expressions on simple scalar values).

If this is not tackled as a bug in EFCore, I think it might be a good idea to extend the documentation to explicitly mention the pitfalls around queries that rely on value-converted fields.

To reproduce, the Sqlite version of the code runs on https://dotnetfiddle.net/ just fine; the SqlServer version needs a DB to connect to.

Your code

// @nuget: Microsoft.EntityFrameworkCore.Sqlite -Version 9.0.1

using System;
using System.Collections.Generic;
using System.Data;
using System.Linq;
using Microsoft.EntityFrameworkCore;

using var db = new BloggingContext();

db.Database.EnsureDeleted();
db.Database.EnsureCreated();

var clientSideEntities = db.MyEntities.ToList().AsQueryable();

// constants work as expected
LogResults(q => q.Where(x => x.MyHex == 16)); // 1(16, 10), 2(16, 16)
LogResults(q => q.Where(x => x.MyDec == 16)); // 2(16, 16)

var myInt = 16;
// parameters work as expected
LogResults(q => q.Where(x => x.MyHex == myInt)); // 1(16, 10), 2(16, 16)
LogResults(q => q.Where(x => x.MyDec == myInt)); // 2(16, 16)

// column-column comparison does not work as expected
LogResults(q => q.Where(x => x.MyHex == x.MyDec));
// expected: 2(16, 16)
// got: 1(16, 10)

// expressions can succeed with unexpected results
LogResults(q => q.Select(x => x.MyHex + 0));
// expected: 16, 16
// got(SqlServer): 256, 256
// got(Sqlite): 10, 10

// expressions can fail on SQL server
LogResults(q => q.Select(x => x.MyHex - 1));
// expected: 15, 15
// got(SqlServer): The data types nvarchar(max) and nvarchar are incompatible in the subtract operator.
// got(Sqlite): 9, 9

void LogResults<T>(Func<IQueryable<MyEntity>, IEnumerable<T>> list)
{
    var expected = list(clientSideEntities).ToList();
    var actual = list(db.MyEntities).ToList();
    PrintList("expected: ", expected);
    PrintList("got: ", actual);
}

void PrintList<T>(string prefix, IEnumerable<T> list)
{
    try
    {
        Console.WriteLine(prefix + string.Join(", ", list.Select(x => x?.ToString())));
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    }
}

public class BloggingContext : DbContext
{
    public DbSet<MyEntity> MyEntities { get; set; }

    protected override void OnConfiguring(DbContextOptionsBuilder options)
        => options
            .LogTo(Console.WriteLine, Microsoft.Extensions.Logging.LogLevel.Information)
            .EnableSensitiveDataLogging()
            //.UseSqlServer("use a valid connection string here");
            .UseSqlite("Data Source=test.db");

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity<MyEntity>()
            .Property(x => x.MyHex)
            .HasConversion(
                v => v.ToString("x"),
                v => int.Parse(v, System.Globalization.NumberStyles.HexNumber)
            );

        modelBuilder.Entity<MyEntity>()
            .Property(x => x.MyDec)
            .HasConversion(
                v => v.ToString(),
                v => int.Parse(v)
            );

        modelBuilder.Entity<MyEntity>().HasData(new MyEntity { Id = 1, MyHex = 16, MyDec = 10 });
        modelBuilder.Entity<MyEntity>().HasData(new MyEntity { Id = 2, MyHex = 16, MyDec = 16 });
    }
}

public class MyEntity
{
    public int Id { get; set; }
    public int MyHex { get; set; }
    public int MyDec { get; set; }

    public override string ToString() => $"{Id}({MyHex}, {MyDec})";
}

Stack traces


Verbose output


EF Core version

9.0.1

Database provider

Microsoft.EntityFrameworkCore.Sqlite

Target framework

.NET 9.0

Operating system

Kali Linux

IDE

Visual Studio Code 1.96.4

EDIT: added one of the relevant issues

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions