Dynamic LINQ Part 2 (Evolution)
A while ago, Scott Gu blogged about a Dynamic LINQ library that was part of the C# samples in VS2008. This library let developers put together late bound LINQ queries using a special syntax. This really comes in handy when you want to build a LINQ query based on dynamic input. Consider the following example:
public ActionResult List(string sort) {
var northwind = new NorthwindEntities();
IQueryable<Product> products = northwind.Products;
switch (sort) {
case "ProductName":
products = products.OrderBy(p => p.ProductName);
break;
case "UnitsInStock":
products = products.OrderBy(p => p.UnitsInStock);
break;
default:
break;
}
return View(products.ToList());
}
In the above example, we’re trying to sort a list of products based on the sort parameter before we render the view. Writing the above code is painful, especially if you have a large number of sortable fields. Of course this isn’t the only way to write this code, in fact, if you’re a LINQ expert you can build an expression tree at runtime that represents the sort expression, but that solution is far from trivial to implement.
Last time we showed how we can (ab)use C# 4 dynamic feature to enable LINQ queries against dynamic types. Today I’ll show you how we can expand on that concept, and turn our previous solution into a general purpose library that allows us to execute late bound queries against existing LINQ providers.
The library itself is pretty small, it consists of 2 classes:
- DynamicExpressionBuilder – The dynamic object that we used last time to build the actual expression tree
- DynamicQueryable – a static class with a bunch of extension methods that are parallel to the actual Queryable static class
DynamicExpressionBuilder uses the same trick as last time (dynamic dispatch to build expressions), but this time we’re building expressions that would be recognized by most LINQ providers.
public override bool TryGetMember(GetMemberBinder binder, out object result) {
result = new DynamicExpressionBuilder(Expression.Property(Expression, binder.Name));
return true;
}
public override bool TryBinaryOperation(BinaryOperationBinder binder, object arg, out object result) {
result = MakeBinaryBuilder(Expression, binder.Operation, arg);
return true;
}
public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result) {
MethodInfo methodInfo = null;
if (args.Any(a => a == null)) {
methodInfo = Expression.Type.GetMethod(binder.Name);
}
else {
var types = args.Select(arg => arg.GetType()).ToArray();
methodInfo = Expression.Type.GetMethod(binder.Name, types);
}
if (methodInfo != null) {
var expression = Expression.Call(Expression, methodInfo, args.Select(Expression.Constant));
result = new DynamicExpressionBuilder(expression);
return true;
}
return base.TryInvokeMember(binder, args, out result);
}
The actual implementation overrides more methods, but for brevity we’re only going to look at 3 of them. Each method captures the intent of the code
and turns it into the appropriate expression tree.
public static class DynamicQueryable {
public static IQueryable Where(this IQueryable source, Func<dynamic, dynamic> predicate) {
return Where(source, GetExpression(source.ElementType, predicate));
}
private static LambdaExpression GetExpression(Type elementType, Func<dynamic, dynamic> expressionBuilder) {
ParameterExpression parameterExpression = Expression.Parameter(elementType, expressionBuilder.Method.GetParameters()[0].Name);
DynamicExpressionBuilder dynamicExpression = expressionBuilder(new DynamicExpressionBuilder(parameterExpression));
Expression body = dynamicExpression.Expression;
return Expression.Lambda(body, parameterExpression);
}
private static IQueryable Where(IQueryable source, LambdaExpression expression) {
return source.Provider.CreateQuery(Expression.Call(typeof(Queryable),
"Where",
new[] { source.ElementType },
source.Expression,
Expression.Quote(expression)));
}
…etc
}
Normally when you’re writing LINQ queries, the implementations of Where, Select, SelectMany and other extension methods come from either the Enumerable or Queryable
class.The DynamicQueryable class uses this pattern to provide overloads of the same extension methods on the non generic IQueryable.
This lets us do a lot of cool things with existing LINQ providers. Let’s look at some examples:
using System;
using System.Linq;
class Program {
static void Main(string[] args) {
NorthwindEntities northwind = new NorthwindEntities();
var query = northwind.Products.Where(p => p.UnitPrice < 10).OrderBy(p => p.ProductName);
foreach (Product p in query) {
Console.WriteLine(p.ProductName);
}
}
}
Above we have a regular LINQ to Entities query against Northwind. Notice the usings are System, and System.Linq, those make sure that you’re using the Queryable
implementations of Where and OrderBy.
We can write this same code in with Dynamic LINQ:
using System;
using DynamicLINQ;
class Program {
static void Main(string[] args) {
NorthwindEntities northwind = new NorthwindEntities();
var query = northwind.Products.Where(p => p.UnitPrice < 10).OrderBy(p => p.ProductName);
foreach (Product p in query) {
Console.WriteLine(p.ProductName);
}
}
}
We replaced the System.Linq using with a DynamicLINQ using, which makes use of our extensions methods. If we wanted to sort by a dynamic variable you could do it
using the indexer syntax supported by DynamicQueryBuilder:
using System;
using DynamicLINQ;
class Program {
static void Main(string[] args) {
NorthwindEntities northwind = new NorthwindEntities();
var query = northwind.Products.Where(p => p.UnitPrice < 10).OrderBy(p => p["ProductName"]);
foreach (Product p in query) {
Console.WriteLine(p.ProductName);
}
}
}
That’s all pretty cool, but there are more scenarios that can’t be done today using regular LINQ which Dynamic LINQ makes easy.
One of the biggest annoyances for me using LINQ today, is the fact that custom functions can’t be used inside of query expressions. This makes total sense once
you understand how LINQ expressions work, but if it were possible, you could make big queries a lot be more maintainable.
Take the following query using LINQ to Entities:
using System;
using System.Linq;
class Program {
static void Main(string[] args) {
NorthwindEntities northwind = new NorthwindEntities();
var query = northwind.Products.Where(p => IsCheap(p));
foreach (Product p in query) {
Console.WriteLine(p.ProductName);
}
}
public static bool IsCheap(Product product) {
return !product.Discontinued && product.UnitsInStock < 10;
}
}
Anyone who has done any programming with LINQ to Entities knows that this won’t work. When we try to run this code, we get an error saying that this expression
cannot be converted to SQL, or more precisely:
Unhandled Exception: System.NotSupportedException: LINQ to Entities does not recognize the method 'Boolean IsCheap(Sample.Product)' method, and this method cannot be translated into a store expression
The reason this doesn’t work is because the compiler puts the IsCheap method call in the expression tree and EF tries to convert it into SQL:
Convert(value(System.Data.Objects.ObjectSet`1[Sample.Product])).MergeAs(AppendOnly).Where(p => IsCheap(p))
What we really wanted is for the compiler to replace IsCheap with the expression in the body of IsCheap, i.e. replace IsCheap with !product.Discontinued && product.UnitsInStock < 10.
This same expression would work today with DynamicLINQ:
using System;
using DynamicLINQ;
class Program {
static void Main(string[] args) {
NorthwindEntities northwind = new NorthwindEntities();
var query = northwind.Products.Where(p => IsCheap(p));
foreach (Product p in query) {
Console.WriteLine(p.ProductName);
}
}
public static dynamic IsCheap(dynamic product) {
return !product.Discontinued && product.UnitsInStock < 10;
}
}
We tweaked the method slightly, changing the argument and return type to be dynamic. When we run this code it works and we see a list of cheap products.
The difference here is that we’re actually executing code to build the expression tree, so instead of putting the method call into the resultant tree we actually invoke the method and the expression tree ends up looking like this:
Convert(value(System.Data.Objects.ObjectSet`1[Sample.Product])).MergeAs(AppendOnly).Where(p => (Not(p.Discontinued) And (Convert(p.UnitsInStock) < 10)))
There are also some scenarios that don’t work in the current implementation. For example, casting and some projections don’t work well with existing LINQ providers. I’ve put the code on a bitbucket repository.
http://bitbucket.org/dfowler/dynamiclinq
Enjoy!