LINQ to SQL Extension: Batch Deletion with Lambda Expression

Batch deletion in the O/R Mapping frameworks is always depressing. We need to query all the entities we want to delete from the database, pass them to the DeleteOnSubmit or DeleteAllOnSubmit methods of DataContext, and finally invoke SubmitChanges to delete the records form database. In this case, we will cost an additional query and send lots of "DELETE" commands to database. How wasteful!

So if we want to make batch deletion with LINQ to SQL, we'll probably execute a SQL command in souce code:

ItemDataContext db = new ItemDataContext();
db.ExecuteCommand(
    "DELETE FROM Item WHERE [CreateTime] < {0}",
    DateTime.UtcNow.AddMonths(-1));

But in my opinion, it's ugly if we put SQL commands in source code. Ideally, the developers should focus on the business logics, data access logics or domain models, and the database operations are encapsulated by some other layers. In the days without O/R Mapping frameworks, we always defines Stored Procedures in databases and make calls in the source code. And finally we get O/R Mapping frameworks such as Hibernate and LINQ to SQL, so in most cases we could write code to query the database without dealing with SQL commands.

But batch deletion is one of the exceptions, as I said minutes ago. To avoid making batch deletion with SQL commands, I build an extension method for LINQ to SQL. Now we can delete the records from database with lambda expression:

ItemDataContext db = new ItemDataContext();
db.Items.Delete(item => item.CreateTime < DateTime.UtcNow.AddMonths(-1));

Of course, we could use more complicated expressions to indicate the entities to delete:

ItemDataContext db = new ItemDataContext();
db.Items.Delete(item =>
    item.CreateTime < DateTime.UtcNow.AddMonths(-1) ||
    item.ViewCount < item.CommentCount && item.UserName != "jeffz"); 

For the extensions I built for LINQ to SQL, most of them are based on DataContext. But now I extended the Table<TEntity> class. The implementation is not so difficult as I thought before I finish it. The key part of extending with LINQ/Lamba Expression is parsing the Expression Tree, but in this case, most of the expressions are binary expression and that's almost all of the extension - I just implemented the common used conditions. For instance, it doesn't support the expressions like "item.Introduction.Length < 10" and it should be converted to the use of LEN function in an completely implemented version.

The "Where Condition" in the SQL command converted from lambda expression is built in 3 steps, you can see the corresponding classes in the source code for more details:

  1. Use PartialEvaluator class to replce the expressions which can be evaluated directly with the constant expressions indicated the calculated results (E.g., replace the expression "3 * 3" with the constant expression of "9", or replace the variables with the values of them).
  2. Use ConditionBuilder class to collect all the constants in the expression tree as parameters for te SQL command, and generate the SQL where condition with the parameters' placeholders in it (E.g., "[CreateTime] < {0} AND [UserName] <> {1}").
  3. Use DataContext.ExecuteCommand method to execute the whole SQL command with parameters and return the number of effected records.

Now we have the function of batch deletion but what about batch updating? Actually I'm working on it. I'm going to let the developers updating the records in the database like this:

ItemDataContext db = new ItemDataContext();
db.Items.Update(
    item => new Item
    {
        Introduction = item.Title + "Hello World",
        ViewCount = item.ViewCount + 1,
    }, // SET [Introduction] = [Title] + 'Hello World', ...
    item => item.CommentCount > 100 /* WHERE condition */);

For more details about the batch deletion extionsion please see the attachment.

Chinese version of this post

Published Thursday, March 06, 2008 4:40 PM by JeffreyZhao

Comments

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Wednesday, March 05, 2008 2:18 PM by Richard

An alternative approach would be to use the GetCommand method of the DataContext, and alter the command text to delete the matching records. That way, any condition supported by LINQ to SQL is supported.

The delete command will need to look something like:

DELETE

FROM TableName

FROM TableName As D0 INNER JOIN (SELECT ....) As D1

ON (D0.Key1 = D1.Key1 And D0.Key2 = D1.Key2 ...)

where "SELECT ..." is the original command text, and the "ON" clause contains all primary key columns for the table. You can find the primary key columns by using reflection on the entity type:

from p in entityType.GetProperties()

from ColumnAttribute a in p.GetCustomAttributes(typeof(ColumnAttribute), false)

where a.IsPrimaryKey

select new

{

   Property = p,

   Column = a

};

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Wednesday, March 05, 2008 9:44 PM by JeffreyZhao

@Richard

That's an interesting idea.

# 扩展LINQ to SQL:使用Lambda Expression批量删除数据。

Wednesday, March 05, 2008 10:19 PM by Zqin

扩展LINQ to SQL:使用Lambda Expression批量删除数据。

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Tuesday, April 01, 2008 1:51 PM by Terry Aney

If there code available for download at all?

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Sunday, April 06, 2008 3:32 AM by Colin B.

Looking at the view from 30,000 feet - Is it only me who thinks that much of Linq is a giant step backward? I mean, comeon, deleting/inserting multiple records is an extremely common task (in my code anyway) and is accomplished using a trivial SQL statement in ADO.NET. This shouldn't be an exercise in academia!

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Monday, April 14, 2008 4:24 AM by Terry Aney

Jeffrey, this article inspired me to create an implementation for both the DELETE and UPDATE.  Have a read over at www.aneyfamily.com/.../Batch-Updates-and-Deletes-with-LINQ-to-SQL.aspx

I also made some references to this article and how I understood it.  If I misrepresented anything, please let me know.

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Monday, July 07, 2008 4:06 AM by Salion

very good.

thanks jeffery zhao.

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Thursday, July 23, 2009 3:09 PM by Thrishna

How does this support using a Delete statement with an "IN" clause.

Like: "DELETE FROM tblMyTable WHERE col1 IN (val1,val2, val3, val4)"

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Tuesday, November 17, 2009 6:02 PM by Terry Aney

Using the extension from my url above, you could do something like...

tblMyTables.DeleteBatch( t => ( new [] { val1, val2, val3, val4 } ).Contains( t.col1 ) );

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Monday, November 30, 2009 3:00 PM by Mitchell

Awesome job - the code is great.

I tried implementing it for v4.0 and got everything working great except for table mapping - evidently EF 4.0 does not have any properties for accessing for table mappings (due to the fact that multiple database tables can be mapped to a single entity).  Any ideas?

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Monday, May 03, 2010 12:44 PM by _ivan

I liked @Richard's idea.

LINQ to SQL already knows how to generate SELECTs T-SQL from lambda expressions.

Why not reuse that to generate WHERE clauses for UPDATEs and DELETEs, and don't forget batch INSERTs too (like:  INSERT INTO (Col1, Col2,...) SELECT Var1 as Col1, Var2 as Col2, ... FROM Table WHERE <Condition>)

Am I completely wrong here?

# LINQ to SQL扩展:用Lambda表达式执行批量删除(Batch Deletion) _ 人猿的巢穴

Pingback from  LINQ to SQL扩展:用Lambda表达式执行批量删除(Batch Deletion) _ 人猿的巢穴

# Batch Updates and Deletes with LINQ to SQL &laquo; Managing My Technical Mess

Pingback from  Batch Updates and Deletes with LINQ to SQL &laquo; Managing My Technical Mess

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Sunday, March 27, 2011 10:57 AM by weblogs.asp.net

Linq to sql extension batch deletion by lambda expression.. OMG! :)

# Use custom SQL statements with LINQ ExecuteQuery with custom classes &laquo; Harvey Darvey&#039;s Blog

Pingback from  Use custom SQL statements with LINQ ExecuteQuery with custom classes &laquo; Harvey Darvey&#039;s Blog

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Wednesday, June 08, 2011 2:29 AM by weblogs.asp.net

Linq to sql extension batch deletion by lambda expression.. Huh, really? :)

# re: LINQ to SQL Extension: Batch Deletion with Lambda Expression

Friday, October 28, 2011 11:39 AM by Martin

@Mitchell said: I tried implementing it for v4.0 and got everything working great except for table mapping - evidently EF 4.0 does not have any properties for accessing for table mappings (due to the fact that multiple database tables can be mapped to a single entity).  Any ideas?

I am having the same problem. Any ideas/workarounds?

Leave a Comment

(required) 
(required) 
(optional)
(required)