Dynamic LINQ (Part 1: Using the LINQ Dynamic Query Library)

LINQ (language integrated query) is one of the new features provided with VS 2008 and .NET 3.5.  LINQ makes the concept of querying data a first class programming concept in .NET, and enables you to efficiently express queries in your programming language of choice.

One of the benefits of LINQ is that it enables you to write type-safe queries in VB and C#.  This means you get compile-time checking of your LINQ queries, and full intellisense and refactoring support over your code:

While writing type-safe queries is great for most scenarios, there are cases where you want the flexibility to dynamically construct queries on the fly.  For example: you might want to provide business intelligence UI within your application that allows an end-user business analyst to use drop-downs to build and express their own custom queries/views on top of data. 

Traditionally these types of dynamic query scenarios are often handled by concatenating strings together to construct dynamic SQL queries.  Recently a few people have sent me mail asking how to handle these types of scenarios using LINQ.  The below post describes how you can use a Dynamic Query Library provided by the LINQ team to dynamically construct LINQ queries.

Downloading the LINQ Dynamic Query Library

Included on the VS 2008 Samples download page are pointers to VB and C# sample packages that include a cool dynamic query LINQ helper library.  Direct pointers to the dynamic query library (and documentation about it) can be found below:

Both the VB and C# DynamicQuery samples include a source implementation of a helper library that allows you to express LINQ queries using extension methods that take string arguments instead of type-safe language operators.  You can copy/paste either the C# or VB implementations of the DynamicQuery library into your own projects and then use it where appropriate to more dynamically construct LINQ queries based on end-user input.

Simple Dynamic Query Library Example

You can use the DynamicQuery library against any LINQ data provider (including LINQ to SQL, LINQ to Objects, LINQ to XML, LINQ to Entities, LINQ to SharePoint, LINQ to TerraServer, etc).  Instead of using language operators or type-safe lambda extension methods to construct your LINQ queries, the dynamic query library provides you with string based extension methods that you can pass any string expression into.

For example, below is a standard type-safe LINQ to SQL VB query that retrieves data from a Northwind database and displays it in a ASP.NET GridView control:

Using the LINQ DynamicQuery library I could re-write the above query expression instead like so:

 

Notice how the conditional-where clause and sort-orderby clause now take string expressions instead of code expressions.  Because they are late-bound strings I can dynamically construct them.  For example: I could provide UI to an end-user business analyst using my application that enables them to construct queries on their own (including arbitrary conditional clauses).

Dynamic Query Library Documentation

Included with the above VB and C# Dynamic Query samples is some HTML documentation that describes how to use the Dynamic Query Library extension methods in more detail.  It is definitely worth looking at if you want to use the helper library in more depth:

 

Download and Run a Dynamic Query Library Sample

You can download and run basic VB and C# samples I've put together that demonstrate using the Dynamic LINQ library in an ASP.NET web-site that queries the Northwind sample database using LINQ to SQL:

You can use either Visual Web Developer 2008 Express (which is free) or VS 2008 to open and run them.

Other Approaches to Constructing Dynamic LINQ Queries

Using the dynamic query library is pretty simple and easy to use, and is particularly useful in scenarios where queries are completely dynamic and you want to provide end user UI to help build them.

In a future blog post I'll delve further into building dynamic LINQ queries, and discuss other approaches you can use to structure your code using type-safe predicate methods (Joseph and Ben Albahari, authors of the excellent C# 3.0 In a Nutshell book, have a good post on this already here). 

Hope this helps,

Scott

45 Comments

  • That´s nice, but how is it handled in LINQ to SQL? Is it possible to use sqlparameters, or does it prevent SQL injection in some other way in senarios like this:

    .Where(String.Format("CategoryID={0}" & Request.QueryString["id"])

  • Thanks Scott, following you up on.

  • Hi Jonatan,

    >>>>>> That´s nice, but how is it handled in LINQ to SQL? Is it possible to use sqlparameters, or does it prevent SQL injection in some other way in senarios like this: .Where(String.Format("CategoryID={0}" & Request.QueryString["id"])

    Because LINQ to SQL uses type-safe data model classes, you are protected from SQL Injection attacks by default. LINQ to SQL will automatically encode the values based on the underlying data type.

    BTW - you can use the Where() extension method either like so:

    .Where(String.Format("CategoryID={0}" & Request.QueryString["id"])

    Or:

    .Where(String.Format("CategoryID=@0", Request.QueryString["id"])

    The second option allows you to specify any parameter markers you want in the query, and then inhect the values as params that you pass separately.

    Hope this helps,

    Scott

  • Thank you Scott!
    I download the C# Dynamic Query Library you gave. I found nice example of Dlinq into path CSharpSamples\LinqSamples\DynamicQuery\DynamicQuery and then I found nice 101 examples into this path CSharpSamples\LinqSamples\SampleQueries about Linq at all. Just run it by VS2008 Express i had all examples by different implementation of linq, like:
    - Linq To Sql
    - Linq to XML
    - Linq over DataSet
    - XQury use cases

    Great!

  • Great! So why is this a zip somewhere on MSDN that has a code file we need to compile? If this works as good as you say it does, please consider putting it in the Extensions release.

    Thanks!

  • erm...
    "One of the benefits of LINQ is that it enables you to write type-safe queries in VB and C#. "

    and then you come with an example which precisely kills this benefit.

    Why not use the example of concatenate Linq queries? (where you use one linq query in another one, which will be combined into 1 expression tree at runtime by the provider?)

  • Interesting stuff!! I'll have a closer look later.

    Is this new? Or has it been around for a while and I just haven't noticed.

  • Hi Frans,

    >>>>>> Why not use the example of concatenate Linq queries? (where you use one linq query in another one, which will be combined into 1 expression tree at runtime by the provider?)

    As I said at the end of this post, my next dynamic linq post will cover using predicate methods to compose LINQ queries. This enables type-safe expression composition.

    Thanks,

    Scott

  • Hi Ben,

    >>>>>>> Interesting stuff!! I'll have a closer look later. Is this new? Or has it been around for a while and I just haven't noticed.

    Believe it or not it has been around awhile. :-) I believe it shipped in the samples in Beta2.

    Thanks,

    Scott

  • Hey Scott,

    Another great post. How's Part 5 of the MVC Series coming?

    Thanks,
    --Steve

  • Using this removes the real time compiler checking for the syntax, isn't it?
    It looks more like Subsonic query or Nhibernate hql......

  • Is it possible to combine a strongly typed Linq expression with a dynamic query?
    For example, create your base query:

    var query = from p in Northwind.Products
    where p.CategoryID = 2
    select p

    and then append a dynamic order by to the query, something like
    query.OrderBy = "SupplierId"

    I prefer to use the strongly typed approach but as you said, not all scenarios allow this (Custom sorting in particular) in which case it would be nice to still do the bulk of your Linq in a strongly typed manor

  • Any Silverlight 2.0 posts in the works?

  • Good things,

    Type-safe and prevent SQL injection.
    I am looking at for your next post.

    Thanks,

  • Hi Scott,
    great I have been waiting for that. Thanks for posting. I look forward for the followups. This might be some sort of-topic. But are you posting about LINQ to SQL and many-to-many relations. How to handel it with LinqDatasource Insert/Update, etc? Or do I have to code this manually and extend my DataContext. Maybe you posted already about that and I am only missing something...
    Anyway thanks for sharing the information...

    Regards

    Felix

  • Hey Now Scott,
    The library's sure are a great resource.
    Thx,
    Catto

  • I have been playing with the Dynamic Query Library for a while now. However, I never could figure out a way to do joins, using this library.

    Thanks,
    Scott

  • What about a scenario where we let our users add their own custom columns to a table (SQL). Can we use Linq to query a table that has changed? Is there some kind of “Refresh” or something of that sort?

  • Scott, I think you meant to have the arguments w/o the call to String.Format like this:

    .Where("CategoryID=@0", Request.QueryString["id"])

    Using String.Format is just a fancier way to concatenate strings and its use can still lead to string injection attacks.

    For example: .Where(String.Format("CategoryID={0}", "'X' OR Secrets=1"))

    When you use the Dynamic API with the '@' parameters and specify arguments directly the two are never concatenated together so your arguments are kept isolated all the way through. A text variables contents will never be misinterpreted as part of the query.

  • Hi Scott,
    Linq is realy great, for me the best thing since Object Oriented Programming (realy!). However, this move i can't follow. Everything you are doing here (and _much_ more) can be done if you have combinable expressions. And that is not hard to implement. So why do you want us to return to the dark ages of glueing strings together to build a query, where you could have a fully typesafe intellisensable solution that would look like:

    if (!String.NullOrEmpty(catselect)) {
    var oldwhereclause=whereclause;
    whereclause = p => oldwhereclause(p) && p.Category.CATEGORYNAME==catselect;
    }
    var q =
    from p in ...
    .Where(whereclause) // don't realy like this syntax here but it works..


    Note that not only my where clause is now constructed typesafe, also the tables needed for the query are changed, which would be troublesome is a string-glueing aproach.

    Cheers, Ferdinand Swaters (still on 2008 Beta 2)

  • Thanks Scott!

    Is it possible to get the following behavior with this library (Like statement "%")? I am having trouble getting it to work.

    Dim _Course = From p In University.Courses _
    Where p.Course_Name.StartsWith("B") _
    Select p

  • Is LINQ to SQL suitable for multi-tier apps? I've read on various sites that LINQ to SQL lacks mulit-tier capabilities. Is this true? Are you still planning a blog to cover multi-tier using LINQ to SQL?

  • Hi Scott,

    Great stuff, but isn't the example dynamic query library already included in System.Web.Query.Dynamic? Any idea why the classes in that namespace are not public?

    Thanks,

    Andrew

  • I don't think your Dynamic LINQ is very good.I think we can use lambda tree to create dynamic LINQ better.

  • What would be sweet is some kind of Predicate textbox that would do Intellisense for the user by understanding the parms, types and operators the dev "sets" for the textbox.

  • Hi Stefan,

    >>>>>> Yesterday I published an article about dynamic Where- and OrderBy-clauses in LINQ to SQL.

    That is a great article. Pointing others at it in case they missed it: http://www.scip.be/index.php?Page=ArticlesNET10&Lang=EN

    Thanks,

    Scott

  • Hi Kris,

    >>>>>>> Is it possible to combine a strongly typed Linq expression with a dynamic query?

    Yes - you can mix both strongly typed queries and the dynamic string based queries together, which is quite nice and useful.

    Thanks,

    Scott

  • Hi Kevin,

    >>>>>>> Any Silverlight 2.0 posts in the works?

    Yep - I am just getting ready to start posting heavily on Silverlight 2.0. :-)

    Thanks,

    Scott

  • Hi Felix,

    >>>>>> great I have been waiting for that. Thanks for posting. I look forward for the followups. This might be some sort of-topic. But are you posting about LINQ to SQL and many-to-many relations. How to handel it with LinqDatasource Insert/Update, etc? Or do I have to code this manually and extend my DataContext. Maybe you posted already about that and I am only missing something...

    I haven't posted on M:M relationships yet. Here is a blog post that discusses it a little though: http://blogs.msdn.com/mitsu/archive/2007/06/21/how-to-implement-a-many-to-many-relationship-using-linq-to-sql.aspx

    Hope this helps,

    Scott

  • Hi Ferdinand,

    >>>>>>> Linq is realy great, for me the best thing since Object Oriented Programming (realy!). However, this move i can't follow. Everything you are doing here (and _much_ more) can be done if you have combinable expressions

    This post is part 1 of 2. In the second part I'm going to discuss using combined expressions and predicate builders to do type-safe query composition.

    Thanks,

    Scott

  • Hi Shloma,

    >>>>>>> Sometime ago you mentioned about doing a post on how to use LINQ in a multi-tier enviroment, I would love to get into LINQ but can't see on how this could work. Would very much appreciate if you could comment on that.

    I still need to write a post on this with LINQ to SQL. It is on my list of things to-do (unfortunately it is a big list though!).

    Sorry!

    Scott

  • Hi Trent,

    >>>>>>> Is it possible to get the following behavior with this library (Like statement "%")? I am having trouble getting it to work.

    You can use .StartsWith(), .EndsWith() and .Contains() to get functionality similar to the like statement with SQL.

    Hope this helps,

    Scott

  • Hi Andrew,

    >>>>>>> Great stuff, but isn't the example dynamic query library already included in System.Web.Query.Dynamic? Any idea why the classes in that namespace are not public?

    Yep - this functionality is included for the LinqDataSource control in ASP.NET. Unfortunately the LINQ team didn't have time to fully design the dynamicquery library to the extent they felt comfortable shipping it as a broadly generic library in the framework with .NET 3.5 - which is why it is currently shipped as a source sample.

    Hope this helps,

    Scott

  • Good article Scott, as always.

    I've been looking at using the CLR in SQL Server to get round problems with dynamic queries in stored procedures. I haven't seen much on the use of LINQ within SQL Server itself. Are you aware of any problems and would I be able to use the Dynamic Query Library there as well?

    Thanks, Simon

  • Thanks Scott, really nice post!!

  • In your code what is NorthwindDataContext?

  • Has anybody tried it on DataTables? I tried it, but I think I did something wrong or it is not supported.

    Thanks for any help!

    Evert

  • Does anyone have any examples of how to use the dynamic query on XML instead of SQL? It would be great if there is an example of how to apply an orderby and allocate the type of the orderby dynamically as well with the source being XML.

  • Hi Scott,

    This is great. I'm glad you've done it, you saved me from updating my dynamic querying library to work with Linq.

    I can't find the namespace of those extension methods though, whey do they live?

    Paymon

  • Dim values = From t1 In MyTables.Table1 _
    JOIN t2 In MyTables.Table2 On t1.ID Equals t2.ID
    Select t1, t2

    How do I dynamically order this query ??

  • "Operator '==' incompatible with operand types 'Guid?' and 'Guid'"

    what am i doing wrong here?

    Guid? SiteID = ((COM_Site)(cmbSectiesDynamic.SelectedItem)).ID;
    List employeeList = this.m_databaseContext.Employees.Where("SiteID ==@0",SiteID).ToList();

    i don't get it..

  • I'm not totally sure if this is a good thing. The great benefit of the 'old' linq is the fact that's it's strong typed. In the early .net days we had typed datasets which is replaced with the much better Linq to Sql. Linq is already providing joining, grouping and filtering.

    I can already do: db.Products.Where(p => p.Category.CategoryName == "Beverages");
    In a production environment 'dynamic' queries don't exist. because all queries are constructed based on contextual arguments. There's nothing dynamic about categoryid (which in your example is filtered on the value '2'). It's not like if I suddenly pass an url parameter SupplierID=5 that's it's included in the query.

    So all query are predetermined. My example can easily changed into db.Products.Where(p => p.Category.CategoryID == Request.QueryString("CatID"));. Using the MVC web application CatID is even processed by controller and passed as a strong (validated) parameter to the data layer (Models).

    Back in the classic ASP days late binding was normal. These days we use strong typed code so the compiler can warn us about any typo's we might have made. Very important for beginning programmers.

    And even linq itself can be used to 'dynamicly' construct a query. Let's say a admin user want to filter the product list. The UI provides him (even better her ;-) which the dropdowns for category, supplier and a textbox for entering minimum unitprice. Most databases only use positive ID's for primary keys, so a returned (form) value of -1 means no selection has been made. UnitPrice has been defined as double? (nullable)

    var products =northwind.Products; //just initializing the quey

    if (categoryId > -1)
    products.Where(p => p.Category.CategoryID == categoryId);

    if (supplierId > -1)
    products.Where(p => p.Supplier.SupplierID == supplierId);

    if (unitPrice != null)
    products.Where(p => p.UnitPrice >= unitPrice);

    DataGrid.DataSource = products;
    DataGrid.DataBind(); //Database query is performed here

    'Dynamic' queries will usually be build in a similar way, except products would than be a string. A string which easily can contains typo's, assiging a number to a text field. Those issues present them selfs only at runtime.

    So again, what's the benefit of using late-bind, non-typed data queries? It's almost like I'm back in the classic ASP days. Like you said, you can rewrite a strong-typed linq query into a late-bind query. But I don't see the point of doing that.

    Maybe you presented me a 'wrong' example of using dynamic linq queries, but for now I only see down sides.

  • How can you handle null values with the LINQ data source control? For example I have a drop down list that controls the data displayed in a gridview.









    With the SQL datasource, I could use "categoryId=@categoryId OR @categoryId IS NULL" as my WHERE clause so that I could show all records when the "blank" item was selected from the drop down or only the specified item category if one was selected.

    With the LINQ datasouce I can't get this to work. If I specifiy "AutoGenerateWhereClause=true" it works when the "blank" item is selected but does not work for the other categories because my categoryId is a GUID. If I specify my own WHERE clause as "categoryId.ToString() = @categoryId" it works fine for everything but the "blank" item.

    Is it poor design on my part or I am missing something with LINQ?

  • Hi scott

    sometimes i want to search the database like this:

    var query=from p in db.Customers
    where p.City.Contains("Lon");

    but,if i change this expression to dynamic linq expression?

    var query=db.Customers.Where("City like @0","Lon"); This dosn't work.

  • I find it annyoing that you can't "refresh" a table block in the Linq to SQL designer window. I mean if I change a field in the DB, then I would like to be able to get that change into the designer. Now I hat do delete the table and then add it again.

Comments have been disabled for this content.