Zeeshan Hirani

Building Expression Trees from Scratch

2008-07-20T03:04:15Z

Just out of curiosity i decided to create expression trees from scratch instead of using lambda statements to get my expressions trees. In all my projects, I haven't really found any significant need to create expression trees from scratch. Most of the time lambda statements are sufficient for me to create any complex expressions. However to understand lambda expressions better, it think it is essential to comprehend, how compiler converts lambda statements to expression syntax. In the example below, I am going to create a very simple expression tree that takes two parameters, multiples both parameters and adds two it.

In the above example, I am first creating a constant expression of type integer. Than I am creating two parameter expressions that my final expression needs. Both parameter expressions are defined as integer type. The third parameter in Expression.Parameter represents the alias that you will use to refer to the parameter in the expression tree. Next I create my first expression which is simply a multiplication of the X and Y parameter. I obtain my final expression by adding the constant with multiplied expression. This is an example where I am combining two expression by using Add operand. Now in order to build my final expression I make use of Expression.Lambda static method that takes the body of the expression which we created earlier follow by the parameters needed by the expression trees.

From the output window, you can see that we have the same output which ever path we choose to create our lambda expressions from. However I think creating even a simple expression tree is a high learning curve. An example such as above which I think is fairly non trivial took me 45 minutes to write. I hope this small tutorial gave you in sight in how to build lambda expressions from scratch.

Modifying Expression Trees

2008-07-19T16:38:55Z

As I have started to play more with expression trees, I decided to learn how to modify expression trees. Well expression trees cannot be modified because they are read only. In order accomplish modification, you have to create a brand new tree, copy the existing expressions that are still valid, replacing the expression that you want changed. Below is an example that demonstrates changing a constant value from 1 to 3.

In the above example, I have a simple expression that adds 6, divides by 2 and adds one. All we are trying to accomplish is change the value of 1 to 3. I am first grabbing the body of the entire expression by calling Body on the expression and casting it to BinaryExpression. A BinaryExpression has a left and right operand. Looking at the graphic view of the expression, you can notice that in order to get to the value of 1, we need to access right side of the expression and cast it to a node type of ConstantExpression. Based on the diagram, I am accessing the right side of the body expression, casting it to ConstantExpression and printing the value of the expression. However as I mentioned earlier, expressions are read only, therefore cannot be changed. When I try to assign new value compiler does not allow. Here is the modified version of the code that replaces the constant expression with new expression.

In the above example I am creating a new expression and grabbing the left portion of the existing expression and replacing the right portion with the new constant expression I have created. You can confirm that the new expression has a value of 3 from the expression tree output printed on the screen.

Just to get a little more complex, I am going to change the value of 6 in the expression tree to 4. Below is an example that illustrates how to achieve the change.

In the above example, in order to get to the value of 6 I have to do two lefts on the binary expressions. Further more I have to recreate all the expressions that are above the expression that I want changed and rebuild the entire tree. From the output, you can confirm that our value of 6 has been updated to new value of 4.

Combining Expression Trees

2008-07-19T05:20:01Z

Today, I was working with expression trees and had the need to combine two expression trees. It appears that when you try to combine two expression trees, it doesn't work. Compiler raises an an exception and does not allow combining two expression trees. Instead you need to convert expression tree into delegate by compiling the expression tree and using the compiled delegate with the existing expression tree to get the new expression tree. Here is an example that C# compiler does not allow.

In the above example, compiler raises an error when I try to merge two expression trees, complaining variable is used like a method. Error is not very meaningful but essentially if you compile the square expression tree into delegate and than use it with squareplus2 expression tree, you get a new expression tree. Here is an updated example of merging two expression trees together.

In the above code, you will notice that I am first compiling my square expression tree to a delegate and than passing in the n parameter expected by the delegate. Later I am getting the ToString version of squareplus2 expression tree which simply prints the expression tree invocation to the output screen. In order to execute my expression tree, I have to first compile it therefore I am compiling my new expression tree, squareplus2 and than passing in the value of 4 to get the value for the entire expression. From the result in the output screen, you can see that our new expression combines both expression to get the correct value.

Different ways of removing elements from XML

2008-07-17T06:36:54Z

Linq to XML provides numerous ways of modifying XML which includes adding, updating and deleting nodes. I happen to work on an XML document from which I had to remove some xelments. Surprisingly I found numerous options that facilitate deleting of various xelements from XML loaded in memory. Depending on where you are in the XML tree, you might find one method easier than the other. Here is a simple example that illustrates various ways I discovered of removing xelements from XML loaded in memory.

In the above example, I have some XML elements nested inside of book element. I start of with deleting title element by first getting access to the title element by using element method. Once I have the instance of the element that I want to remove I simply call remove on the element to remove the element from the tree.

Next, I am deleting ISBN number by calling SetElementValue. Basically SetElementValue takes an element and the value for the element. If the element does not exists, it gets added. If element is present in the tree, the value gets updated. However if the value is set to null, the element gets removed from the tree.

In the last option, I am removing the totalchapters element by first getting a reference to the element and calling the replaceWith method passing null for the element parameter. ReplaceWith method takes an element that you want the existing element to be replaced with. Passing value of null indicates that you want to simply remove the existing element.

From the output, you can confirm that title, ISBN and totalchapters elements are removed from the book node and therefore not printed on the output screen.

XElement.ToString returns encoded text

2008-07-17T05:19:22Z

I am sure people who do XML on a day to day basis would not find surprising but I think it was pretty cool to see that when I call ToString on my XML, I get back my entire XML with whitespace preserved and also all text is encoded for me automatically. I created my XML using .net 2.0 api and saw the same behavior so I guess this was not a new feature but definitely a good default option to encode your text before outputting it to the user. Here is simple example that illustrates this behavior.

As you can see from the above code, my title and author value have some html tags that gets encoded when i output the content to the console.

Avoid impurities in Linq To SQL

2008-07-15T05:49:19Z

This problem actually troubled me for hours until I accidentally figured out how to get around it. What I learned is try to avoid impurities in Linq statements. For instance, if you lambda expression's filter is applied by indexing an array value, do not access the array directly inside the lambda value. Instead get the value from the array assign it to a variable and use the variable in the Linq to SQL filter statements. I am not sure why I have to declare a variable to avoid an impurity. But if you do not follow this technique, more than likely you would end up with runtime error in your Linq statement. Even if you get pass the runtime error, you would end up with a wrong filter value being applied on your lambda expressions. Let's take a look at an example to understand the problem I am trying to describe.

In the above example I grab the top 2 order for every city in my array list which happens to be London and New York. In order to accomplish that I am looping through my array of cities and for each city, I am creating a lambda expression filtered by city. On the next iteration of the loop I am going to the else statement and combining my existing Iqueryable of orders with the old one. Basically in terms of SQL, I am simply applying Union operator to union Order records for both cities. However if the query were to execute the sql generated has city filter that set to New York as shown in the diagram. We would have expected that first part of the query would have the filter for the city of London and next part of the query would have a filter of New York. However in this case both portions of the query had a filter of New York. This is the side effect of accessing cities array directly inside of a lambda expression. Since we accessed cities array directly, the last value the array had on the last iteration of the loop, is the value that Linq to sql uses for all the filters in the query.

In order to fix the issue simply grab the value from the array and store it in a variable and use that variable inside of the lambda expression. Here is the correct version of the query that applies the correct filter.

Applying aggregates to empty collections causes exception in Linq To SQL

2008-07-15T04:55:20Z

I spent nearly two hours trying to figure out what was wrong with my Linq to SQL query. Basically I was trying to calculate the sum of sales generated by each sales agent. In order to accomplish that I had to travel from employee table to Orders which had the employeeid linked to it. From the Order table I navigated to its order details and calculated the sum of Quantity ordered multiplied by Unit Price. The query works fine when each agent has put in an order in the database. But when there is no order record for an agent the sum operation performed on SQL server returns null. When Linq to SQL tries to assign null values to data types that is not defined as null, it throws an exception. Below is an example that demonstrates this issue.

In the above query, I am displaying the Employee's First name and for each employee getting its orders. For every order, I grab its order details and than flatten all order details for all orders for a particular customer using SelectMany operator. Once I have all the OrderDetails, I apply sum operator operator to get TotalRevenue generated by each employee. The above query causes exception because there are some employees that do not have any orders and Linq to SQL raises exception complaining that null value cannot be assigned to a member with type System.Decimal which is non-nullable. What this means is in our lambda expression od.UnitPrice and od.Quantity is defined as decimal and short where as employees that dont have orders and thus does not have any order details, the value for unit price and quantity is null. Since you cannot assign null value to a data type that is non-nullable, Linq to SQL throws an exception.

To solve the problem, simply do an explicit cast to nullable type and your query would execute fine. Here is the corrected version of the query written in two different ways. Both ways require an explicit cast to nullable type.

In the above example, I have two different queries that return the same result. I was simply trying to force Linq to SQL to generate two different query plans and sure enough modifying the query generates unique SQL queries but they both met the met same fate and caused exception unless you do a correct cast to nullable type.

Entity Refs Not getting serialized with WCF service

2008-07-14T05:09:35Z

In one of the project that I am working on, my client requires me to use WCF service to talk to Linq To SQL datacontext. The infrastructure guys would not open up port for me to access SQL server directly. Therefore I created a WCF project in my solution, added a reference to Business library and exposed my Linq to SQL entities using WCF service. By default you cannot expose your Linq to SQL entities, you have to mark SerializationMode on the datacontext to Unidirectional. Once you configure the datacontext for wcf serialization, the generated entities are marked with DataContract attribute and properties on the entity are marked with Datamember attribute. If Linq to SQL entity have child entities those also get serialized. However child entities do not get lazy loaded, you have to explicitly call LoadWith to immediately load child collections for them to be serialized. Here is a simply example that illustrates the behavior.

In the above screen shot, I am marking the serialization mode on the data context to Unidirectional which causes entity classes to be serialized using WCF service. In the above example, I also have a WCF service that exposes an operation called GetOrder. GetOrder method simply retrieves an order based on orderid passed. Since we want the OrderDetails to be also available on the client and serialized along with the Order, I am loading OrderDetails ahead of time.

In the above example, I am creating an instance of my wcf service and getting the order from the service. After retrieving the order, I am printing the orderid and the total of all the orderdetails for the order. But when I access the Customer property of the Order object I don't see any Customer property available, although my Linq to SQL entities had Customer property on my order object. For some reason, the Serialization mode does not take care of serializing entity refs.

I am still researching the problem. If anyone had seen this behavior and knows a workaround to the problem, please write a comment to let others know.

Automatic entityset mapping in Entity Framework

2008-07-14T01:23:31Z

As I am starting to do work on both linq to SQL and linq to entities, I am finding lots of subtle differences on both implementations. Some implementations I would consider as being better over the other.

In Linq to SQL, if I have a customer object and want to get access to its orders collection, all I simply have to do is navigate to the Orders collection property and linq to SQL lazy loads the orders for that particular customer instance. But what happens if I already have orders in the memory and some of the orders do actually belong to the customer that is loaded. Although using primary keys, linq to SQL can associate those orders to the customer based on the mapping definition but it will not do so. However in entity framework, if you have loaded some orders from the database randomly based on some criteria and if there is a customer in memory whose orders are part of that randomly selected orders, than entity framework would automatically associate those orders to that customer with out you making a request for the Orders for that customer. It could be considered a plus and minus point. Plus because it automatically uses the associations of the customer with orders based on entity key to tie those orders to that customer. The minus is because those subset of orders in memory may not represent all the orders for that customer and that could be very confusing and hard to debug problems when the Orders Collection does not show all the orders for the customer. Here is an example that illustrates this issue.

In the above example, I am turning off deferred loading on linq to SQL datacontext. The reason I am doing is because when I apply the count operator on the Orders collection, I dont want linq to SQL to go and fetch the orders for the customer instance. In the linq to SQL portion of the code, I am first retrieving all the orders in the database and loading it up in the memory. I then go ahead and retrieve the ALFKI customer from the database and print its Orders. Based on the results on the output window of 0, you can surmise that linq to SQL did not associate the orders in the tracking service with customer based on the primary key value.

Next portion of my code does the same thing except using entity framework. I first retrieve the customer and than I am retrieving all the orders for the employee id of 4. Next I am printing the Orders for the ALFKI customer. By default we would expect a value of 0 based on how entity framework is implemented because in entity framework the child collections do not get loaded until you call Load method. However we still see 2 in the output window. The reason is, the orders we loaded earlier based on employeeid of 4, has 2 orders that belong to ALFKI customer. This result is misleading because there are actually 6 orders of ALFKI customer but only 2 of them are loaded in memory.

Eager Loading child entities in Entity Framework

2008-07-13T22:36:50Z

Once again I am comparing different behaviors in linq to SQL and entity framework. If you want to eagerly load child entities in Linq to SQL, you would use LoadWith operator available on DataLoadOptions. LoadWith method retrieves related objects for the main object. You have the option to call LoadWith method as many times as you need depending upon how many related objects you want to eagerly load. If you want to eagerly load related objects in entity framework, you you can call include method. Include takes a path which represents the name of the related entity that you want to load. In the path parameter you can also use traversal by using dot notations. For instance for a particular customer, if you want to load its Orders and for each order want to load its OrderDetails, instead of making of two Include calls you can call include with a traversal like cust.Include("Orders.OrderDetails"). This tells entity framework to load Orders and for each Order load its OrderDetails as well. As with linq to SQL, you also can make more than 1 calls to Include to load multiple entities at the same time. Below is an example that illustrates different usage of eager loading with entity framework and linq to SQL.

In the above example, I start off with showing how to eagerly load child collections with linq to SQL. I am using DataLoadOptions class and calling LoadWith method to load its Orders and for each order load its Order Details. I am also loading the addresses for the customer as well. Notice that I had to make use of LoadWith method 3 times but all my code is strongly type and I am not having to type in strings in the load with parameter. Instead I make use of lambda expressions to specify what sub collections to load.

Now coming back to entity framework, in order to eagerly load sub collections I am using Include operator. Notice in order to load order and its order detail, I am not calling Include operator several times. Instead I am traversing the entity path which tells entity framework that I want to load order and its order details. As I mentioned earlier, you have the option to call Include several times, therefore I am calling Include again to load addresses for the customer as well. Over all I think Include syntax is better than Linq's LoadWith except Include is not strongly type meaning my string does not evaluate until runtime and I dont get design time compilation which is offered by LoadWith operator in Linq to SQL.

Below is the output from the above query. The results from linq to SQL and entity framework is the same.

Lazy Loading child entities in Entity Framework

2008-07-13T21:04:34Z

It's pretty interesting to see different implementations of lazy loading in both linq to SQL and entity framework. In linq to SQL, if you want to access the Orders collection of a particular customer, all you do is navigate to the Orders Collection Property and linq to SQL will run the SQL in the background and fetch the orders for that customer instance. The operation is pretty implicit. However in entity framework you have to explicitly tell that I intent to load the orders of the customer. The way you would do this is by calling Load on the Orders collection of the customer. If you forget to call load on the Orders Collection for the customer, you would end up with an empty collection of orders. This may be confusing and hard to troubleshoot bug and you would wonder why you have no orders for that customer when you could see orders in the database for the customer. Here is an example that illustrates this behavior.

In the above code, I am fetching the orders for alfki customer and printing it to the console. All that was required was simply to navigate to the Orders property and linq to SQL runs the query in the background and gets the orders for the customer. When I try to do the same thing with entity framework, it does not work. You can see that first time I try to print the orders for the customer using entity framework, I get a 0 count and that is because I have not explicitly told entity framework that I want orders for that customer. Next time around I call Orders.Load which ensures that orders for the customer are loaded and sure enough after that when I access the orders collection I get 6 Orders.

Not All Lambdas can be converted to expression trees!

2008-07-10T05:24:25Z

It was surprising for me to discover that depending upon how you write your lambda syntax, it may not be able to convert to expression tree. If you are using a body expression in your lambda statements meaning you don't have an explicit return statement and no parentheses, than the lambda expression can be converted to expression tree. However lambda expression with statement body which requires an explicit return statement can only be converted to delegates. Expression trees are important because most providers of linq take expression trees and convert them to their domain specific language where as delegates gets compile into IL code which only Linq To Objects provider can understand. This is why it is important that your lambda statements are convertible to expression trees. Here is an example below which illustrates a lambda statement that cannot be converted to expression tree and the error compiler raises in regards to that.

Different Ways of representing Lambdas

2008-07-10T04:49:40Z

Today I had some free time at work since my batch process was taking an hour to run. During this time, I was investigating the beauties of lambda expression. I have found lambda expressions to be very flexible and versatile. What I mean is, your lambda expression can be as long and as small as you need it. If you find that in certain portion of your code, it would be much more easier to read if there is less noise from language, than you can use type inference. Places where you feel the code would be easier to read if there is more explicit types declared, should have proper parentheses and return statement than by all means using statement syntax form of lambda expression. If you find the lambda expressions getting polluted by too much code and thus making it harder to read than by all means point the lambda expression to a method that does all the complicated logic. Here are different ways I have been able to express lambda statements.

In the above code, you would notice that i have expressions that have no data type and return statements where as some expressions have explicit data type and return statement. I also have a lambda expression that simply points to a method. Basically you can make lambda expressions as complex and as simply as you want it to be.

Compiling Extension Methods on .net 2.0 runtime

2008-07-08T05:22:53Z

Today, I had to work on a legacy .net 2.0 application and I found a neat little trick one of developers did to get extension methods to work on .net 2.0 runtime. I was pretty amazed because I thought extension methods was a feature available in System.Core assembly that is part of .net 3.5 framework. In reality that is completely true and Extensions are available only in .net 3.5. However extension methods gets converted into the same IL code that is compatible with .net 2.0 runtime. So the only thing left is to satisfy the Visual Studio compiler that extension methods are valid statements for .net 2.0. On asking the developer who wrote the code, he said that in Visual studio .net 2.0 compiler only checks for an attribute called System.Runtime.CompilerService.ExtensionAttribute to be available for extension methods to work. So he basically added a dummy class of that type that inherits from Attribute class. After adding the ExtensionAttribute class, the compiler was satisfied and the project build fine.

The above code does not compile because the project is targeting .net 2.0 assemblies. Once we add our dummy class called System.Runtime.CompilerService.ExtensionAttribute the project builds correctly as shown in the screen below.

Design Time Compilation in Vs 2008 Sp1

2008-07-05T22:17:53Z

I have been playing around with Vs 2008 sp1 bits and one of the very cool debugging improvements I have noticed is, you don't have to compile the project to see compilation errors. By default C# compiler would put squiggly lines below the code that is causing the problems and when you hover over the squiggly lines it will show the exact error that you would get if you were to compile the project. I think this feature offers a tremendous developer productivity. I cant imagine how many times have I actually compiled the project to see what error the compiler gives me. But now with this feature I will save so much time. All I have to do is hover over the lines that is causing the problem I will get the same error as if I was compiling the project. Hats off to C# team in pulling off this feature. Screen shot below shows this feature in action.