May 2008 - Posts
When using LINQ, queries may bloat up to dozens of lines. My personal style is to take these queries and break them apart to smaller units of logic. To each unit of logic, I append a call to ToArray. @yosit asked me why I did it and I answered I was avoiding a possible pitfall. Here's what I meant.
Take the following code for instance:
static void Main(string[] args)
{
int[] arr = { 1, 2, 3, 4, 5, 6, 7, 8, 9 };
var filter = from n in arr
where VeryLongOperation(n)
select n;
var cartesian = from n in filter
from m in filter
select 2 * n + 3 * m;
var result = cartesian.ToArray();
}
Imagine VeryLongOperation only allows numbers up to 3 and prints the number of times it was called to the debug output. It looks as though the very long operation will run only once per number, so you'll only have three calls, but here's the actual output you get in the debug window once you run this code:
Operation #1
Operation #2
Operation #3
...
Operation #36
This is caused by LINQ's deferred execution, which means that every time an item is taken from any of the loops, it will go back to the first filter and call the very long operation again. This means that you have 3 calls that cause 9 inner loops (27 calls together) and 6 that don't cause inner loop calls. 3 + 27 + 6 = 36.
Let's make a slight alteration to the original filter query:
var filter = (from n in arr
where VeryLongOperation(n)
select n).ToArray();
This forces LINQ to execute the query now. Sure, there's a slight overhead of creating a new array, but it's a static array, so that mitigates the problem a bit. Now the debug window looks like this:
Operation #1
...
Operation #9
This is a neat trick and is one of the first things that I look for when reviewing code with multiple LINQ statements.
First of all, if you've come here looking for how to activate connection pooling when using SSAS 2005 via ADOMD.NET, you're in for a little surprise - there is none by design.
Loading a new connection can take up a long time, since with every new session to the database, all of the metadata and security context has to get loaded too. I've decided to perform some mental gymnastics and try and implement a simple connection pooling mechanism. Here is the (very) basic version of the mechanism, which I may continue to post changes to as they accumulate:
public static class AdomdConnectionPool
{
private static Dictionary<string, Dictionary<string, ArrayList>> pool =
new Dictionary<string, Dictionary<string, ArrayList>>();
/// <summary>
/// Gets a connection from the pool or creates one if one does not exist.
/// </summary>
public static PooledConnection GetConnection(string connectionString)
{
// Pooling (See the Poll method)
ValidateListExistance(connectionString);
return GetConnectionFromPool(connectionString);
}
private static PooledConnection GetConnectionFromPool(string connectionString)
{
KeyValuePair<string, ArrayList> session =
new KeyValuePair<string, ArrayList>(null, null);
lock (pool[connectionString])
{
if (pool[connectionString].Count > 0)
{
// Available session exists. Use it.
session = pool[connectionString].First();
pool[connectionString].Remove(session.Key);
}
}
// No available connections exist. Create a new one.
if (session.Key == null)
return CreateNewConnection(connectionString);
// A session exists
AdomdConnection pooledConnection = new AdomdConnection(connectionString);
pooledConnection.SessionID = session.Key;
try
{
pooledConnection.Open();
// Register the session with the pool.
PooledConnection poolItem = new PooledConnection(pooledConnection, session.Value);
poolItem.Disposed += new Action<PooledConnection, EventArgs>(poolItem_Disposed);
return poolItem;
}
catch (Exception ex)
{
// Connection probably expired. Try again.
return GetConnectionFromPool(connectionString);
}
}
private static void ValidateListExistance(string connectionString)
{
lock (pool)
{
if (!pool.ContainsKey(connectionString))
pool.Add(connectionString, new Dictionary<string, ArrayList>());
}
}
private static PooledConnection CreateNewConnection(string connectionString)
{
// Create a new connection and register it with the pool.
PooledConnection poolItem = new PooledConnection(
new AdomdConnection(connectionString),
new ArrayList());
poolItem.Disposed += new Action<PooledConnection, EventArgs>(poolItem_Disposed);
poolItem.Connection.Open();
return poolItem;
}
private static void poolItem_Disposed(PooledConnection sender, EventArgs e)
{
Dictionary<string, ArrayList> connections = pool[sender.Connection.ConnectionString];
try
{
// Close the connection, but keep the session alive.
sender.Connection.Close(false);
lock (connections)
{
// Reclaim the connection to the pool.
connections.Add(sender.Connection.SessionID, sender.ExtraData);
}
}
catch
{
// Can't close connection? Don't let it back in the pool.
// We don't really care why, though. If necessary in the future, log.
}
}
}
Each new connection has a SessionID property, which is unique and is given to each connection that is opened without an existing value in the SessionID property. If there is a value in the property before the connection opens, the connection connects to that session once Open is called. It may have been wiser to use the same SessionID for all connections, but since timeouts may happen on old connections (see the end of the poolItem_Disposed method), I decided to use many, fresh connections. I admit I have no idea whether it's a best practice, but I have yet to see evidence otherwise.
Each pooled connection has two properties we need to use: Connection and ExtraData:
public class PooledConnection : IDisposable
{
/// <summary>
/// Creates a new instance of <see cref="PooledConnection" />.
/// </summary>
internal PooledConnection(AdomdConnection connection, ArrayList extraData)
{
this.Connection = connection;
this.ExtraData = extraData;
}
public AdomdConnection Connection { get; private set; }
public ArrayList ExtraData { get; private set; }
void IDisposable.Dispose()
{
if (this.Disposed != null)
this.Disposed(this, EventArgs.Empty);
}
internal event Action<PooledConnection, EventArgs> Disposed;
}
The ExtraData property is there to store connection-specific data, such as a list of session-scoped members already created at runtime on this session, etc.
Using the mechanism is very simple and resembles using a simple ADO.NET connection:
using (PooledConnection pooledConnection = AdomdConnectionPool.GetConnection(connectionString))
{
// Execute the query
AdomdDataAdapter adapter = new AdomdDataAdapter(query, pooledConnection.Connection);
}
Please note that you can not use this sort of connection pooling when you're using the integrated role-based security, unless you save your sessions according to roles. This has not and will not be implemented in my connection pooling mechanism and if you need it, you'll have to write one for yourself.
I would love to hear comments about the whole thing. Remember that it's pretty basic, but I would love to make it more interesting and intricate and most of all - useful.
This is a short, off-topic rant. Please bare with me.
In an effort to spark some life into my home laptop (which is about four years old and fighting like a champ), I've gone through the list of services running on my machine to try and stop those that are non-essential to me.
To my surprise, I found XobniService.exe on the list of running processes. Since I had previously uninstalled Xobni and hadn't restarted since, I thought a restart would clear it up. It didn't.
To me, that seemed odd: I have none of your products installed, but your update service remains installed on my machine? To what avail? Not to mention that there was no way to uninstall it except to directly use InstallUtil!
I am writing this rant because this is neither the first, nor the last application to do this. This commonplace practice of leaving behind applications that call home, after the original application has been completely uninstalled is annoying - even if you're not sending any sensitive data, your application is still using my computer's resources (CPU, hard drive, Internet connection, etc.) post-mortem.
If you're writing an application that has an updater, make sure to uninstall that too once your application is uninstalled.
Thank you.
As I do from time to time, here is a batch of three Extension Methods I've written recently:
IndicesWhere
/// <summary>
/// Gets the indices where the predicate is true.
/// </summary>
public static IEnumerable<int> IndicesWhere<T>(this IEnumerable<T> enumeration, Func<T, bool> predicate)
{
// Check to see that enumeration is not null
if (enumeration == null)
throw new ArgumentNullException("enumeration");
// Check to see that predicate is not null
if (predicate == null)
throw new ArgumentNullException("predicate");
int index = 0;
foreach (T item in enumeration)
{
if (predicate(item))
yield return index;
index++;
}
}
This is especially useful when you want to cache indices from an array, rather than the array itself. Here's an example:
var indicesWithValues = values.IndicesWhere(value => value != null);
TakeEvery
/// <summary>
/// Take items from 'startAt' every at 'hopLength' items.
/// </summary>
public static IEnumerable<T> TakeEvery<T>(this IEnumerable<T> enumeration, int startAt, int hopLength)
{
// Check to see that enumeration is not null
if (enumeration == null)
throw new ArgumentNullException("enumeration");
int first = 0;
int count = 0;
foreach (T item in enumeration)
{
if (first < startAt)
{
first++;
}
else if (first == startAt)
{
yield return item;
first++;
}
else
{
count++;
if (count == hopLength)
{
yield return item;
count = 0;
}
}
}
}
This is equivalent to an unbounded series of Skip(startAt).Take(1).Skip(hopLength).Take(1).Skip(hopLength)... Useful for when you, for instance, need only every other item in a list.
Distinct
It's really been pissing me off that there's no overload to Distinct that takes a delegate, which means I have to write a new class whenever my comparison isn't the default one. When talking about Anonymous Types, Distinct becomes useless. So here's an overload I can actually use:
private class EqualityComparer<T> : IEqualityComparer<T>
{
public Func<T, T, bool> Comparer { get; internal set; }
public Func<T, int> Hasher { get; internal set; }
bool IEqualityComparer<T>.Equals(T x, T y)
{
return this.Comparer(x, y);
}
int IEqualityComparer<T>.GetHashCode(T obj)
{
// No hashing capabilities. Default to Equals(x, y).
if (this.Hasher == null)
return 0;
return this.Hasher(obj);
}
}
/// <summary>
/// Gets distinct items by a comparer delegate.
/// </summary>
public static IEnumerable<T> Distinct<T>(this IEnumerable<T> enumeration, Func<T, T, bool> comparer)
{
return Distinct(enumeration, comparer, null);
}
/// <summary>
/// Gets distinct items by comparer and hasher delegates (faster than only comparer).
/// </summary>
public static IEnumerable<T> Distinct<T>(this IEnumerable<T> enumeration, Func<T, T, bool> comparer, Func<T, int> hasher)
{
// Check to see that enumeration is not null
if (enumeration == null)
throw new ArgumentNullException("enumeration");
// Check to see that comparer is not null
if (comparer == null)
throw new ArgumentNullException("comparer");
return enumeration.Distinct(new EqualityComparer<T> { Comparer = comparer, Hasher = hasher });
}
I'll be integrating these methods into the Linq Extensions project soon enough. Good hunting. :)
Tamir hates lambdas. He was having a problem with one of his lambda expressions and twittered about it. Around that time I opened my twitter account (yes, Yosi finally convinced me) and offered my help.
He wanted to have a single extension method that could iterate over a collection and either change or keep the values it got. Kind of like this:
items.ForEach(item => item.SubItems.ForEach(subItem => subItem = newValue));
Where ForEach was defined as:
static void ForEach<T>(this IList<T> collection, Action<T> action)
{
for (int i = 0; i < collection.Count; i++)
{
action(collection[i]);
}
}
This did not work simply because subItem is a local variable - a copy of either the reference (when using a Reference Type) or of the value itself (when using a Value Type) - the lambda expression was (very loosely) translated to the following:
void Foo(SubItem subItem)
{
subItem = newValue;
}
What I then tried to do was to replace Action<T> with a delegate with a ref parameter, so that the local variable will change. It turns out lambda expressions can not contain ref parameters.
On to the next solution - overloading the ForEach extension method:
static void ForEach<T>(this IList<T> collection, Action<T> action)
{
for (int i = 0; i < collection.Count; i++)
{
action(collection[i]);
}
}
static void ForEach<T>(this IList<T> collection, Func<T, T> action)
{
for (int i = 0; i < collection.Count; i++)
{
collection[i] = action(collection[i]);
}
}
This way, one can now write the following:
items.ForEach(item => item.SubItems.ForEach(subItem => newValue));
The first call (outer ForEach) will use the overload with Action<T> and the second (inner ForEach) will use the Func<T, T> overload and end up changing the current subItem. The C# compiler does a nice job finding the best overload to call.
More Posts