Contents tagged with LINQ
This is a common request, and really makes sense; we need to use LINQ expressions and a bit of reflection magic. First, an helper function for returning an expression that points to a member:
Then, we call it on the DbContext.OnModelCreating method, as a parameter to StructuralTypeConfiguration<TStructuralType>.Property:And that’s it!
Some days ago I wrote a post on comparing LINQ expressions where I shared my discoveries and general discontent on how difficult it is do right. The reason I was looking into it was because I wanted to write a LINQ query caching mechanism.
There are actually three main problems involved:
- Comparing two LINQ expressions;
- Caching a LINQ expression for some time;
- For cached instances, preventing them from executing when they are enumerated after the first time.
Comparing LINQ Expressions
I ended up writing my own comparer, which involved looking at all possible types of Expressions (around 20 classes). For some it is quite easy, because they have few relevant properties – take ConstantExpression, for instance, I only had to consider the Value property – while for others there are a few – BinaryExpression has Left, Right, Conversion and Method. OK, so here is the code:
This implementation disregards the lambda variable name, so that “x => …” is equal to “y => …”. Also, as you can see, this is not safe for multithreaded usage, because it uses an accumulator field (hashCode), where the hash for the expression currently being calculated is stored. Being an IEqualityComparer<Expression>, it implements both Equals and GetHashCode methods.
As you can see, we are free to supply out own ObjectCache implementation, provided we place a IServiceProvider implementation (UnityServiceLocator will do) on the ObjectCache.Host property and this implementation returns a valid ObjectCache instance. Feel free to replace this by any other similar mechanism!
Preventing Multiple Query Executions
So, when an IQueryable<T> is first executed, it will go to the database, or someplace else (just think WCF Data Services’ DataServiceQuery<T>), and return its results. If we are going to put that query in a cache, we want to prevent it from executing multiple times, otherwise the purpose of the cache would be defaced. For that, I built my own class that just inherits from IQueryable<T> (actually, from IOrderedQueryable<T>, for support of ordered queries) and overrides the IEnumerable<T> (of which IQueryable<T> descends) GetEnumerator method:
Putting It All Together
This allows me to write code as this:
By calling the AsCacheable extension method, our LINQ queries get cached for the specified duration. This will work with any LINQ implementation.
Remember those old posts on Dynamic LINQ? You are probably aware that Microsoft has made its implementation available as a Nuget package, but, like I said, you already have it in your machine, hidden inside the System.Web.Extensions assembly.
In order to make it easier to use, I wrote a simple extension method that works with plain old IQueryable<T>. And here it is:
It even supports parameters! Just two simple examples – I am using Entity Framework, but you can use whatever you like, this is totally generic:
To make it clear, all parameters must be indicated as @0, @1, and so on. It is better to use them, because the database engine can reuse the execution plan.
There’s one limitation, though: you can’t compare each value on its own, you have to specify one of its properties. For example, you can’t have:
The @it parameter is not recognized, which is a pity.
Make sure you have references to both System.Web and System.Web.Extensions, this is required, and won’t affect anything.
As always, glad to be of service!
I recently came upon this problem: how to calculate a hash from a LINQ expression so that I can safely compare two expressions for equality? I had naively assumed that the Expression class – and their descendants – would have implemented GetHashCode in an appropriate way, so as to make developer’s lifes easier, but unfortunately Microsoft thought otherwise.
After looking it up on the Internet, I could see two “solutions”:
- Convert the Expression to its string representation and get the hash code of it;
- Use an ExpressionVisitor to visit all contained expressions and calculate their individual hash – if this seems recursive to you, that’s because it is!
Comparing the String Representation
The first “solution” doesn’t actually work, because two expressions might represent exactly the same and yet have different string representations. For example, consider:
The only thing that differentiates these two expressions is the name of the lambda parameter, unfortunately it causes their string representations to be different.
One possible solution might be to use a regular expression to get all occurrences of lambda expressions, capture the name of the lambda variables, and then do a search and replace for some well known name:
At first sight – at least, for me! – this seemed to work, however, the replacement pattern – “get me all words composed of only the lambda variable” - might match something that it wasn’t supposed to, for instance:
I might use a different replacement regular expression, for example, I could check for “all lambda variables followed by a dot (.)”:
But this wouldn’t get code like this:
To call it off, I think it might be possible, but it is more complicated than it seems.
Using an ExpressionVisitor
The next approach involves using an ExpressionVisitor to visit all expressions contained in an Expression, something that I have talked before. The thing here is, an Expression is typically composed of several other expressions, of various kinds, so appropriate code must be used for each in order to get an univocal hash out of it. For example, for a ConstantExpression, we might consider its NodeType, Type and Value properties. The problem is, there are more than 20 Expression subclasses, and we need to do this for each.
To cut things short, suppose we have implemented an hash code extractor for each Expression kind. We can’t simply have a big hash code by adding all hash codes calculated for each Expression fragment, because the order by which they appear is also important:
So, each individual hash code must be take in order, and then we must get the hash code of the entire list of hash codes. Uff!
This problem exists since there are LINQ expressions, and, of course, other folks have come up with solutions. The NHibernate project has one, probably LINQ to SQL and Entity Framework have too, just to name those more close to me. However, I don’t think any of these solutions might be ready for external, general purpose usage outside their source projects. It would be great to have one such library that we could pick up and use, but I haven’t so far found any. Might be something to work on, though.
What are your thoughts?
A common request when working with LINQ queries (Entity Framework, NHibernate, etc) is the ability to intercept them, that is, inspect an existing query and possibly modify something in it. This is not extremely difficult to do “by hand”, but Microsoft has a nice class called ExpressionVisitor which makes the job easier. It basically has virtual methods that get called whenever the class visits each expression contained in a greater expression, which may come from a query (the IQueryable interface exposes the underlying Expression in its Expression property). The virtual methods even allow returning a replacement for each expression found, the only problem is that you must subclass ExpressionVisitor to make even the slightest change, so I wrote my own class, which exposes all node traversal as events, one event for each kind of expression, where you can return an alternative expression, thus changing the original query. Here is the code for it:
Yes, I know, I probably should have used properties instead of events, but that’s really not important.
A simple example might be:
As always, hope you find it useful!
This is a short post to complement my previous one on Entity Framework Code First Inheritance: how to query for a specific class. The options are:
- From the DbContext collection:
- From the inner ObjectContext, using Entity SQL and the OFTYPE operator:
- Also in Entity SQL, using TREAT and IS OF:
- Using LINQ to Entities:
- Using pure SQL:
Since this blog started, back in 2008, I wrote a lot of posts. I’d say some are still up to date. I picked a few of them, those I’m more proud of, in no particular order.
ASP.NET Web Forms:
Let me know what you think of them! Are there others you particularly enjoyed?
To my great satisfaction, LINQ is nowadays present everywhere, from XML processing, to database querying, including SharePoint. The “Q” in it standing for Query, it’s not a surprise that is mainly being used for querying, but it would be interesting to use it for other scenarios, such as updating and deleting.
This is part of a series of posts about NHibernate Pitfalls. See the entire collection here.
Entity Framework Code First makes it very easy to access local (first level) cache: you just access the DbSet<T>.Local property. This way, no query is sent to the database, only performed in already loaded entities.