mardi 10 août 2021

Why runtime Expressions cause collisions on the Cache of Entity Framework Core 5?

Before I forget it, my execution context, I'm using .Net 5 with the packages:

  • Microsoft.EntityFrameworkCore.Design 5.0.6
  • Microsoft.EntityFrameworkCore.Relational 5.0.6
  • MySql.EntityFrameworkCore 5.0.3.1

My main goal was to remove the repetitive task of doing expressions when I need to retrieve entities, something like:

public class GetListEntity
{
   property int QueryProperty { get; set }
}

public class Entity
{
   property int Property { get; set }
}

public async Task<ActionResult> List(GetListEntity getListEntity)
{
   var restrictions = new List<Expression<Func<Entity>
   if (model.QueryProperty != null)
   { 
      restrictions.Add(e => e.Property == model.QueryProperty);
   }
   ... //Add all the queryable properties and Aggregate the Expression on the variable expectedEntity and get an IQueryable of the entities as queryableEntities from my DbContext
   
   var nonTrackedQueryableEntities = queryableEntities.Where(expectedEntity)
                                                      .AsNoTracking();

   // I will get the total first because the API was meant to paginate the responses.
   var total = await entites.CountAsync();
}

I've managed to achieve what I wanted but let's say... partially, because if I try to Query the Database at least two times in a row I get this exception:


System.ArgumentException: An item with the same key has already been added. Key: e
   at System.Collections.Generic.Dictionary`2.TryInsert(TKey key, TValue value, InsertionBehavior behavior)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareLambda(LambdaExpression a, LambdaExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareBinary(BinaryExpression a, BinaryExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareLambda(LambdaExpression a, LambdaExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareUnary(UnaryExpression a, UnaryExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareExpressionList(IReadOnlyList`1 a, IReadOnlyList`1 b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.CompareMethodCall(MethodCallExpression a, MethodCallExpression b)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.ExpressionComparer.Compare(Expression left, Expression right)
   at Microsoft.EntityFrameworkCore.Query.ExpressionEqualityComparer.Equals(Expression x, Expression y)
   at Microsoft.EntityFrameworkCore.Query.CompiledQueryCacheKeyGenerator.CompiledQueryCacheKey.Equals(CompiledQueryCacheKey other)
   at Microsoft.EntityFrameworkCore.Query.RelationalCompiledQueryCacheKeyGenerator.RelationalCompiledQueryCacheKey.Equals(RelationalCompiledQueryCacheKey other)
   at MySql.EntityFrameworkCore.Query.Internal.MySQLCompiledQueryCacheKeyGenerator.MySQLCompiledQueryCacheKey.Equals(MySQLCompiledQueryCacheKey other)
   at MySql.EntityFrameworkCore.Query.Internal.MySQLCompiledQueryCacheKeyGenerator.MySQLCompiledQueryCacheKey.Equals(Object obj)
   at System.Collections.Concurrent.ConcurrentDictionary`2.TryGetValue(TKey key, TValue& value)
   at Microsoft.Extensions.Caching.Memory.MemoryCache.TryGetValue(Object key, Object& result)
   at Microsoft.Extensions.Caching.Memory.CacheExtensions.TryGetValue[TItem](IMemoryCache cache, Object key, TItem& value)
   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult](Object cacheKey, Func`1 compiler)
   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.ExecuteAsync[TResult](Expression query, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, Expression expression, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ExecuteAsync[TSource,TResult](MethodInfo operatorMethodInfo, IQueryable`1 source, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.CountAsync[TSource](IQueryable`1 source, CancellationToken cancellationToken)'
   

Following the trace I managed to discover that the ORM is caching for some reason my expressions (and putting the parameter name, in this case 'e') and failing to detect a key collision the second time it has a similar expression to query the database. I said for some reason because, it's not the main deal but at least is odd that cache is involved in a non tracked query, maybe I'm missing something in the middle.

To undenrstand how i got here i will put the code below.

First an interface to implement in every model related with querying a list of entities and expose the extension method ListRestrictions (almost at the bottom).

public interface IEntityFilter<TEntity>
{ 
}

The next step was to define Attributes to summarize the action to do with the property and generate a partial expression to use in the extension method:

[AttributeUsage(AttributeTargets.Property, AllowMultiple = false)]
    public abstract class FilterByPropertyAttribute : Attribute
    {
        protected string FirstPropertyPath { get; }

        protected IEnumerable<string> NPropertyPath { get; }

        public FilterByPropertyAttribute(string firstPropertyPath, params string[] nPropertyPath)
        {
            this.FirstPropertyPath = firstPropertyPath;
            this.NPropertyPath = nPropertyPath;
        }

        protected MemberExpression GetPropertyExpression(ParameterExpression parameterExpression)
        {
            var propertyExpression = Expression.Property(parameterExpression, this.FirstPropertyPath);
            foreach (var propertyPath in this.NPropertyPath)
            {
                propertyExpression = Expression.Property(propertyExpression, propertyPath);
            }
            return propertyExpression;
        }

And to avoid comparisons with nullable structs


        public abstract Expression GetExpression(ParameterExpression parameterExpression, object propertyValue);
    }

    public abstract class NonNullableValuePropertyFilterAttribute : FilterByPropertyAttribute
    {
        public NonNullableValuePropertyFilterAttribute(string firstPropertyPath, params string[] nPropertyPath)
            : base(firstPropertyPath, nPropertyPath)
        {
        }

        public override Expression GetExpression(ParameterExpression parameterExpression, object propertyValue)
        {
            var propertyExpression = this.GetPropertyExpression(parameterExpression);
            return this.GetExpression(propertyExpression, this.GetConvertedConstantExpression(propertyExpression, Expression.Constant(propertyValue)));
        }

        protected abstract Expression GetExpression(MemberExpression memberExpression, UnaryExpression unaryExpression);

        private UnaryExpression GetConvertedConstantExpression(MemberExpression memberExpression, ConstantExpression constantExpression)
        {
            var convertedConstantExpression = Expression.Convert(constantExpression, memberExpression.Type);
            return convertedConstantExpression;
        }
    }

An Attribute with a defined role would be:


public class EqualPropertyFilterAttribute : NonNullableValuePropertyFilterAttribute
    {

        public EqualPropertyFilterAttribute(string firstPropertyPath, params string[] nPropertyPath)
            : base(firstPropertyPath, nPropertyPath)
        {
        }

        protected override Expression GetExpression(MemberExpression memberExpression, UnaryExpression unaryExpression)
        {
            return Expression.Equal(memberExpression, unaryExpression);
        }
    }

And last, the extension itself:

    public static class EntityFilterExtensions
    {
        public static List<Expression<Func<TEntity, bool>>> ListRestrictions<TEntity>(this IEntityFilter<TEntity> entityFilter)
        {
            var entityFilterType = entityFilter.GetType();            
            var propertiesInfo = entityFilterType.GetProperties()
                                                 .Where(pi => pi.GetValue(entityFilter) != null 
                                                              && pi.CustomAttributes.Any(ca => ca.AttributeType
                                                                                                 .IsSubclassOf(typeof(FilterByPropertyAttribute))));

            var expressions = Enumerable.Empty<Expression<Func<TEntity, bool>>>();
            if (propertiesInfo.Any())
            {
                var entityType = typeof(TEntity);
                var parameterExpression = Expression.Parameter(entityType, "e");
                expressions =  propertiesInfo.Select(pi =>
                {
                    var filterByPropertyAttribute = Attribute.GetCustomAttribute(pi, typeof(FilterByPropertyAttribute)) as FilterByPropertyAttribute;
                    var propertyValue = pi.GetValue(entityFilter);
                    var expression = filterByPropertyAttribute.GetExpression(parameterExpression, propertyValue);
                    return Expression.Lambda<Func<TEntity, bool>>(expression, parameterExpression);
                });
            }

            return expressions.ToList();
        }
    }


A usage would be:


public class GetListEntity : IEntityFilter<Entity>
{
   [EqualPropertyFilter(nameof(Entity.Property))]
   property int QueryProperty { get; set }
}

public class Entity
{
   property int Property { get; set }
}

public async Task<ActionResult> List(GetListEntity getListEntity)
{
   var restrictions = getListEntity.ListRestrictions();
   ... //Add all the queryable properties and Aggregate the Expression on the variable expectedEntity and get an IQueryable of the entities as queryableEntities from my DbContext
   
   var nonTrackedQueryableEntities = queryableEntities.Where(expectedEntity)
                                                      .AsNoTracking();

   // I will get the total first because the API was meant to paginate the responses.
   var total = await entites.CountAsync();
}

And to be discarded, if I Aggregate a non dynamic expression of a list of expressions, the ORM works fine, when I do it with the dynamic ones I get the exception at the beginning.

I found a workaround, changing in the method extension this line:


var parameterExpression = Expression.Parameter(entityType, "e");

For this one:


var parameterExpression = Expression.Parameter(entityType, $"{entityType.Name}{entityFilter.GetHashCode()}");

I wanna know why this happens and maybe if there is another way to fix it. I posted here before opening a thread in any Github repository because I'm still curious if is a fault of mine for missing something in the way or a bug.





Aucun commentaire:

Enregistrer un commentaire