New "Orcas" Language Feature: Lambda Expressions

Last month I started a series of posts covering some of the new VB and C# language features that are coming as part of the Visual Studio and .NET Framework "Orcas" release.  Here are the first two posts in the series:

Today's blog post covers another fundamental new language feature: Lambda Expressions.

What are Lambda Expressions?

C# 2.0 (which shipped with VS 2005) introduced the concept of anonymous methods, which allow code blocks to be written "in-line" where delegate values are expected.

Lambda Expressions provide a more concise, functional syntax for writing anonymous methods.  They end up being super useful when writing LINQ query expressions - since they provide a very compact and type-safe way to write functions that can be passed as arguments for subsequent evaluation.

Lambda Expression Example:

In my previous Extension Methods blog post, I demonstrated how you could declare a simple "Person" class like below:

I then showed how you could instantiate a List<Person> collection with values, and then use the new "Where" and "Average" extension methods provided by LINQ to return a subset of the people in the collection, as well as compute the average age of people within the collection:

The p => expressions highlighted above in red are Lambda expressions.  In the sample above I'm using the first lambda to specify the filter to use when retrieving people, and the second lambda to specify the value from the Person object to use when computing the average.

Lambda Expressions Explained

The easiest way to conceptualize Lambda expressions is to think of them as ways to write concise inline methods.  For example, the sample I wrote above could have been written instead using C# 2.0 anonymous methods like so:

Both anonymous methods above take a Person type as a parameter.  The first anonymous method returns a boolean (indicating whether the Person's lastname is Guthrie).  The second anonymous method returns an integer (returning the person's age).  The lambda expressions we used earlier work the same - both expressions take a Person type as a parameter.  The first lambda returns a boolean, the second lambda returns an integer. 

In C# a lambda expression is syntactically written as a parameter list, followed by a => token, and then followed by the expression or statement block to execute when the expression is invoked:

params => expression

So when we wrote the lambda expression:

p => p.LastName == "Guthrie" 

we were indicating that the Lambda we were defining took a parameter "p", and that the expression of code to run returns whether the p.LastName value equals "Guthrie".  The fact that we named the parameter "p" is irrelevant - I could just have easily named it "o", "x", "foo" or any other name I wanted.

Unlike anonymous methods, which require parameter type declarations to be explicitly stated, Lambda expressions permit parameter types to be omitted and instead allow them to be inferred based on the usage.  For example, when I wrote the lambda expression p=>p.LastName == "Guthrie", the compiler inferred that the p parameter was of type Person because the "Where" extension method was working on a generic List<Person> collection.

Lambda parameter types can be inferred at both compile-time and by the Visual Studio's intellisense engine (meaning you get full intellisense and compile-time checking when writing lambdas).  For example, note when I type "p." below how Visual Studio "Orcas" provides intellisense completion because it knows "p" is of type "Person":

Note: if you want to explicitly declare the type of a parameter to a Lambda expression, you can do so by declaring the parameter type before the parameter name in the Lambda params list like so:

Advanced: Lambda Expression Trees for Framework Developers

One of the things that make Lambda expressions particularly powerful from a framework developer's perspective is that they can be compiled as either a code delegate (in the form of an IL based method) or as an expression tree object which can be used at runtime to analyze, transform or optimize the expression. 

This ability to compile a Lambda expression to an expression tree object is an extremely powerful mechanism that enables a host of scenarios - including the ability to build high performance object mappers that support rich querying of data (whether from a relational database, an active directory, a web-service, etc) using a consistent query language that provides compile-time syntax checking and VS intellisense.

Lambda Expressions to Code Delegates

The "Where" extension method above is an example of compiling a Lambda expression to a code delegate (meaning it compiles down to IL that is callable in the form of a delegate).  The "Where()" extension method to support filtering any IEnumerable collection like above could be implemented using the extension method code below:

The Where() extension method above is passed a filter parameter of type Func<T, bool>, which is a delegate that takes a method with a single parameter of type "T" and returns a boolean indicating whether a condition is met.  When we pass a Lambda expression as an argument to this Where() extension method, the C# compiler will compile our Lambda expressions to be an IL method delegate (where the <T> type will be a Person) that our Where() method can then call to evaluate whether a given condition is met.

Lambda Expressions to Expression Trees

Compiling lambdas expressions to code delegates works great when we want to evaluate them against in-memory data like with our List collection above.  But consider cases where you want to query data from a database (the code below was written using the built-in LINQ to SQL object relational mapper in "Orcas"):

Here I am retrieving a sequence of strongly typed "Product" objects from a database, and I am expressing a filter to use via a Lambda expression to a Where() extension method. 

What I absolutely do not want to have happen is to retrieve all of the product rows from the database, surface them as objects within a local collection, and then run the same in-memory Where() extension method above to perform the filter.  This would be hugely inefficient and not scale to large databases.  Instead, I'd like the LINQ to SQL ORM to translate my Lambda filter above into a SQL expression, and perform the filter query in the remote SQL database.  That way I'd only return those rows that match the query (and have a very efficient database lookup). 

Framework developers can achieve this by declaring their Lambda expression arguments to be of type Expression<T> instead of Func<T>.  This will cause a Lambda expression argument to be compiled as an expression tree that we can then piece apart and analyze at runtime:

Note above how I took the same p=>p.LastName == "Guthrie" Lambda expression that we used earlier, but this time assigned it to an Expression<Func<Person, bool>> variable instead of a Func<Person,bool> datatype.  Rather then generate IL, the compiler will instead assign an expression tree object that I can then use as a framework developer to analyze the Lambda expression and evaluate it however I want (for example, I could pick out the types, names and values declared in the expression). 

In the case of LINQ to SQL, it can take this Lambda filter statement and translate it into standard relational SQL to execute against a database (logically a "SELECT * from Products where UnitPrice < 55").

IQueryable<T> Interface

To help framework developers build query-enabled data providers, LINQ ships with the IQueryable<T> interface.  This implements the standard LINQ extension method query operators, and provides a more convenient way to implement the processing of a complex tree of expressions (for example: something like the below scenario where I'm using three different extension methods and two lambdas to retrieve 10 products from a database):

For some great blog post series that cover how to build custom LINQ enabled data providers using IQueryable<T>, please check out these great blog posts from others below:

Summary

Hopefully the above post provides a basic understanding of how to think about and use Lambda expressions.  When combined with the built-in standard query extension methods provided in the System.Linq namespace in "Orcas", they provide a really rich way to query and interact with any type of data while preserving full compile-time checking and intellisense. 

By taking advantage of the Expression Tree support provided with Lambdas, and the IQueryable<T> interface, framework developers building data providers can ensure that the clean code that developers write executes fast and efficiently against data sources (whether a database, XML file, in-memory object, web-service, LDAP system, etc).

Over the next few weeks I'll complete this language series covering the new core language concepts from a theoretical level, and then move on to cover some super practical examples of using them in action (especially using LINQ against databases and XML files). 

Hope this helps,

Scott

Published Sunday, April 8, 2007 4:21 PM by ScottGu
Filed under: , , ,

Comments

# re: New "Orcas" Language Feature: Lambda Expressions

Sunday, April 8, 2007 9:08 PM by $

The code becomes unreadable in Reflector, not very useful :(

# re: New "Orcas" Language Feature: Lambda Expressions

Sunday, April 8, 2007 9:57 PM by ScottGu

Hi $,

I believe Lutz has a new version of Reflector out that handles extension methods now.

Thanks,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 3:38 AM by Ibleif

Is this possible with VB also?

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 3:43 AM by ScottGu

Hi Ibleif,

You'll be able to use Lambdas from VB in "Orcas" as well.  I don't believe support for this is in Beta1, but it will be added for Beta2 and the final release.

Thanks,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 9:45 AM by Greg

One of the best, concise, descriptions of Lambda's that I've seen yet.  Thanks.

One question that I can't seem to find the answer to.  How does the more friendly syntax:

from p in people

where p.Name = "Guthrie"

select p.Name

get translated to the extension methods & Lambdas:

people.Where(p => p.Name == "Guthrie").Select(p => p.Name)

Is this just compiler magic or is it another language feature?

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 10:13 AM by Vikram

just another very good post by Scott. Love reading them

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 1:48 PM by Mike

"In the case of LINQ to SQL, it can take this Lambda filter statement and translate it into standard relational SQL..."

Does it take into account differences between for instance SQL Server 2000, 2005 or MySQL? If so, how? Paging for instance, is complicated in SQL Server 2000, was added to 2005 and has been dead easy in MySQL for years. Do we need to worry about this?

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 1:51 PM by Michael

Interesting post. I believe I will be using it without having to know the ins and outs, but still: Expression<Func<Person, bool>>

What happens here? I know about List<T> but this syntax I have never seen before. Can you elaborate (or link to an article that does)? Thanks!

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 9, 2007 4:47 PM by Peter

How would a subquery look like using LINQ to SQL?

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 6:03 AM by Mark

A great concise explanation, but nothing I've yet read tells me how to SAY 'p => p.LastName == "Guthrie"' out loud!

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 8:25 AM by Andy

Awesome Scott.. absolutely awesome! Now if only I can convince our department heads to not run screaming from the thought of another Visual Studio/.NET Framework upgrade when Orcas hits!! Time to work on my evangelism skills ;)

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 9:07 AM by Josh Stodola

(yet again) Another greatly detailed post, Scott!  Just a personal question, how much time do you spend on these blog posts?  It appears like alot.

Is anybody out there brave enough to convert this to VB?

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 11:37 AM by ScottGu

Hi Greg,

My next blog post in this series is going to cover the new LINQ query syntax, and explain how it gets translated into Extension Methods with Lambdas.

I wanted to get a separate blog post out on Extension Methods and Lambdas out first, so that I could reference those in this upcoming one.

Stay tuned!

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 11:38 AM by ScottGu

Hi Mike,

A data provider implementing IQueryably could determine what type of database it is going against, and automatically adjust the SQL under the covers to implement paging as efficiently as possible.

Hope this helps,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 11:42 AM by ScottGu

Hi Josh,

In terms of how long it takes me to write a blog post >> it depends a lot on the post.  

I can bang out ASP.NET or IIS ones pretty quickly (1-2 hours for tutorials).  

For language ones like this series they can take me longer.  Extension Methods and Lambdas are two of the topics that - if presented incorrectly - can really confuse people.  I ended up spending about 5-6 hours this past weekend writing the above post - and tried a few different approaches to the post to find one that I thought was both concise/crisp, but also approachable.

I'm hoping now that I've got lambdas out of the way, the next few posts will start to get easier (since I can just refer back to this one for more details on lambdas).

Thanks,

Scott

P.S. I am planning to covert this series to VB once I'm done (and can find a few free hours).

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 12:56 PM by Jimmy

Hi Scott,

Any idea when the next CTP / Beta of Orcas will be released (for download)?

Thank you.

Regards,

Jimmy

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 2:21 PM by ScottGu

Hi Jimmy,

Orcas Beta1 should be out later this month, and will be a public download for everyone to use.

Not too far now. :-)

Thanks,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 10, 2007 5:49 PM by Jeff

Is this the beginning of the end of SQL?

(drooling at the thought)

# re: New "Orcas" Language Feature: Lambda Expressions

Wednesday, April 11, 2007 10:41 AM by Josh Stodola

Thanks for the reply - I was going to guess this article took you at least 4 hours.  Would have taken me an eternity.  Those time consuming efforts are most definitely reflected in your writing.  An absolutely superb final draft every time!!

Thanks so much, again, for dedicating this much time to your blog, and keep up the excellent work.

# re: New "Orcas" Language Feature: Lambda Expressions

Thursday, April 12, 2007 1:53 PM by David Parslow

There has been a lot of talk about ajax, linq, and the new designer with Orcas but is there any discusion about improvements made to the asp.net (like the improvements and additions made to controls in asp.net 2.0)?

# re: New "Orcas" Language Feature: Lambda Expressions

Friday, April 13, 2007 1:46 AM by ScottGu

Hi David,

I'll be talking more about the ASP.NET runtime improvements in Orcas in the future.  There are a number of new controls that are really cool (asp:listview, asp:linqdatasource, etc).  ASP.NET AJAX is also built-in with Orcas.

From a programming model perspective, I think LINQ and the ORM data providers that support it are some of the biggest innovations - which is why I'm doing this series to help provide some of the language background on how to really take advantage of it.

Hope this helps,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Friday, April 13, 2007 9:38 PM by Sam Stange

I'm glad you are putting a lot of effort into LINQ. This really pushes .NET to a new level! It's difficult to create pages where you need to search a database table with a dynamic where clauses without writing a ton of code and having to worry about SQL injection issues.

In addition, there are so many O/R mappers out there! LINQ really helps to solve this issue. I've played around with SubSonic, and got a taste of a transparent DAL, and it left me wanting more. A lot of people may not think much of it now, but this is a huge timesaver! Great job!

# re: New "Orcas" Language Feature: Lambda Expressions

Sunday, April 15, 2007 4:53 PM by misro

scott,

Thanks for the very well written article. It was a very easy read. Looking forward to following more of your articles.

misro

# re: New "Orcas" Language Feature: Lambda Expressions

Monday, April 16, 2007 3:03 PM by jc

Pretty awesome. Though the syntax could have been EVEN better:

people.Where(|p| p.LastName == "Guthrie")

:-)

# re: New "Orcas" Language Feature: Lambda Expressions

Saturday, April 21, 2007 12:37 PM by ScottGu

Hi Ben,

Sorry for the delay in getting back to you (for some reason I missed your comment).

You'll definitely continue to see ASP.NET evolve and get richer and richer.  There are a lot of cool new features (including great LINQ support) coming as part of the "Orcas" release.  It is always going to be the server-side web programming model for .NET.

Silverlight (which contains "WPF/E") is also going to provide a ton of richness for the client-side web programming model for .NET.  

Developers will be able to use ASP.NET and Silverlight separately if they want - but you'll also see a lot of integration between the two (allowing you to build one solutions that integrates features from both).

We'll be talking a lot more about this starting next week at our MIX conference.

Hope this helps,

Scott

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 24, 2007 2:56 AM by Marlon Grech

Hi Can you compile .net 3.0 applications with Orcas or does it only compile .Net 3.5?

# re: New "Orcas" Language Feature: Lambda Expressions

Tuesday, April 24, 2007 10:58 AM by ScottGu

Hi Marlon,

Yes - Orcas supports "multi-targetting", which means you can build .NET 2.0, .NET 3.0 and .NET 3.5 applications using VS Orcas.

Hope this helps,

Scott