Omer van Kloeten's .NET Zen

Microsoft Israel's Developers' Academy #3

Omer van Kloeten — Tue, 09 Sep 2008 12:47:28 GMT

Voting has been open for a few days now for which lectures will be featured at the next Microsoft Developers' Academy. The event is the second most important event for Microsoft-platform-developers in Israel, surpassed only by the biennial Tech-Ed Israel.

Among some very interesting lectures I would love to see, I've suggested a lecture myself. Here's its abstract:

A Deep Dive into LINQ

Many developers already use LINQ on a daily basis, but most only scratch the surface of what’s possible. In this session we will dive more deeply into LINQ and see how it works behind the scenes, share tips, tricks and common pitfalls.

While most of the lecture's structure is already defined in my head, I'd love to hear from you what topics you would like to see covered, either from the topics covered in my blog's posts or from your own experience.

Go vote and I'll be seeing you there! :)

Visual Studio 2008 Load Testing Checklist

Omer van Kloeten — Tue, 09 Sep 2008 12:33:50 GMT

After a couple of days of trying to run a load test for a web service on several agents via Visual Studio 2008, I come out much wiser and with a few new bald-spots, where hair I pulled out in the process used to be.

I got a few errors whose messages have nothing to do with what really happened, so here's a quick checklist:

Try restarting the agent service.
Re-register the agent using AgentConfigUtil and then restart the agent service.
Try restarting the controller service.
Are you using a trial version of Visual Studio Team System? If so, it may have expired.
Are all of your agents and controller on the same version?
Check that the file Microsoft.VisualStudio.QualityTools.LoadTestFramework.dll is the same version on all of your computers (more information). Its original version is 9.0.21022.8, SP1's version is 9.0.30729.1. If you installed the trial from Microsoft's site, your version is pre-SP1.
If any of the agents differs from the controller, you will not be able to use them and will see them as Disconnected in the Administer Test Controllers dialog.
Are you using SQL Server as a store for the results?

Does the user the service runs as have permissions to the SQL Server database?
You can see which user is used to access the store from Visual Studio's Menu Test -> Administer Test Controllers.
If you're getting login errors, check that the SQL Server allows remote connections. If it does, consider using a SQL Server user instead of Windows Authentication. Remember to make sure the SQL Server allows both types of connections (Mixed Mode) before attempting this.
Make sure the computer from which the SQL Server is running has an exception for incoming connections on TCP port 1443.

Using the trial version of the agents? Getting "The Visual Studio Team Test Load Agent license has expired" way before your 90 day trial period has elapsed? You've probably hit the 25 tests mark (note that the tests are per-CPU, rather than per-computer).
You can work around this if you change the user running the service.

Around that time I decided I've had enough with all this trial stuff. I was thinking about purchasing licenses, but then I took a look and found that you can only use load agents when you've got a Volume License!

At that point, I went and just ran the test on a few computers at once.

Extending The Partial Classes of Stored Procedures' Results in LINQ to SQL

Omer van Kloeten — Thu, 04 Sep 2008 09:31:00 GMT

One of the nice things about LINQ to SQL is the ability to extend the types of the generated entities. Another nice thing is being able to get typed results from stored procedures. Let's try and combine the two together.

We'll take the following generated code for instance:

public partial class GetStuffResult
{
    
    private int _Id;
    
    private string _Name;
    
    public GetStuffResult()
    {
    }
    
    [Column(Storage="_Id", DbType="INT NOT NULL")]
    public int Id
    {
        get
        {
            return this._Id;
        }
        set
        {
            if ((this._Id != value))
            {
                this._Id = value;
            }
        }
    }
    
    [Column(Storage="_Name", DbType="NVarChar(100)")]
    public string Name
    {
        get
        {
            return this._Name;
        }
        set
        {
            if ((this._Name != value))
            {
                this._Name = value;
            }
        }
    }
}

I would like to extend it by adding a new property to it, named TranslatedId, which translates the returned Id to and from a Guid:

partial class GetStuffResult
{
    public Guid TranslatedId
    {
        get
        {
            return Translator.GetGuid(this.Id);
        }
        set
        {
            this.ProjectId = Translator.GetInt(value);
        }
    }
}

Translator's two methods translate between the integer id and the Guid id. Note that 0 translates to and from an empty Guid.

I'll try to run it and will find out that all of my Ids are 0. Why is that?

Apparently, when running a stored procedure in LINQ to SQL, it requires all of the properties defined on the class to have both getters and setters (which means you can not write read-only properties in the partial class) and sets all non-result (i.e. non-generated) properties to their default values. This means that although the real Id gets selected from the database, TranslatedId gets set to an empty Guid immediately afterwards, which in turn overrides Id to 0.

To work around this, we have to apply a dirty little hack:

partial class GetStuffResult
{
    private Guid dupe;

    [Column(Storage = "dupe")]
    public Guid TranslatedId
    {
        get
        {
            return Translator.GetGuid(this.Id);
        }
        set
        {
            this.ProjectId = Translator.GetInt(value);
        }
    }
}

This is pretty horrible, but what it does is direct LINQ to SQL to set the default value into the property's storage field, which is some dupe field, instead of into the property itself. Now the field's value will be reset and not the property's. Note that the dupe field's type should be the same as the property's type.

Yes, it's ugly, but this is the only workaround I found. If anyone knows of any other workaround or solution to this problem, I'd love to hear it.

[Original image used: Where do we go next?]

Announcing New C# 4.0 LINQ Features and Book

Omer van Kloeten — Fri, 29 Aug 2008 10:12:00 GMT

* Fabrice Marguerie, co-author of LINQ In Action, asked me to clarify that this post has absolutely nothing to do with their book and is simply meant in jest *

LINQ has been around for quite a while, making our lives easier with its short, declarative syntax. During talks in closed sessions, the persons behind C# have come up with some great plans for its next version, including some vast improvements to LINQ. The team wanted to make everyone's lives even better by introducing some ground-breaking features to it.

I'd like to introduce, out of those finalized or at the final draft stage, some of the new keywords that have been introduced to the language with regards to LINQ and by doing so, expose the reader to this new, exciting technology.

fine

Many of the lines of code incorporated into LINQ are about variable validation. How many times have you had to write pieces of code like this:

var values = from n in numbers
             where n > 0 && n < 10
             select n;

This piece of code is very trivial and the compiler is smart enough to understand it on its own. From C# 4.0, the fine keyword is introduced and the above query will be replaced with:

var values = from n in numbers
             where n is fine
             select n;

some

Most LINQ queries do not iterate over an entire enumeration. Some filter out results, some use Take, Skip and other methods in order to run on only a subset of the elements. Since this is an extremely common scenario, the team has decided to integrate it into the language using the new some keyword:

var values = from n in some numbers
             select n;

This is complemented with the new most keyword, which is like some, only it takes more elements.

somewhere

The from clause is known to all who use LINQ, as it indicates the source of the data. Since this clause is a recurring pattern (statistical surveys have shown it to occur 99% of the time, allowing for a 4% standard deviation), the team behind the new language features has decided to cut it out, thus removing unnecessary verbosity from the language. This, in turn, will cause less characters to be written and rewritten, meaning less keystrokes, less keyboard wear, less purchases of new keyboards, leading to prevention of global warming, a goal near to Microsoft's heart. You can see Microsoft's predictions for this in the graph to the right.

This is possible due to the new somewhere keyword, as described below:

var values = from somewhere
             select n;

This feature relies heavily upon the newly introduced Type Inference Transfer System (abbreviated MANTA), a technology recently demonstrated by Microsoft Research.

The Book

I've long since wanted to write a book about something. Since this is just the tip of the iceberg, I've decided to write my book about this subject. It will contain most of what I know (hey, I still need to speak about something at conferences and release a second edition some day, right?) about the new language. I think that with all of the new features, making our lives as developers that much easier, I've picked an appropriate title.

Here's an excerpt from the introduction:

Recent advances in compilation technologies have brought with them less need for verbosity. In its stead come smart compilers, which infer much of the work for the developer. With such technology, there's not a lot you, as a developer, need to do anymore. You can simply write a short statement, validate its correctness, check the code in and go home for the day.

Welcome to the world of tomorrow. Put your feet up.

Here's the layout of the book, as I plan it:

Introduction
New Features in LINQ

What else would you like to see in the book? Let me know and I might just send you a copy of the rough draft for review once I finish it.

Throw Before You Yield

Omer van Kloeten — Mon, 25 Aug 2008 19:23:00 GMT

In a comment left by Bart De Smet, he pointed out that I failed to address the fact that the execution of all "yielding" methods is deferred.

For instance, when running the following code, no exceptions will be thrown:

int[] arr = null;
var copy = arr.Enumerate();



// ...

static IEnumerable<T> Enumerate<T>(this IEnumerable<T> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    foreach (var item in enumeration)
    {
        yield return item;
    }
}

This is because the first time the code from the method will run will be after there a call to copy's GetEnumerator(), followed by a call to that object's MoveNext() method has been made (i.e. when we enumerate over copy).

To overcome this problem, we'll change the code slightly:

static IEnumerable<T> Enumerate<T>(this IEnumerable<T> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    return EnumerateCore(enumeration);
}

private static IEnumerable<T> EnumerateCore<T>(IEnumerable<T> enumeration)
{
    foreach (var item in enumeration)
    {
        yield return item;
    }
}

Now a NullReferenceException will immediately be thrown out of Enumerate, since the "unyielding" method will first run the code and then defer-call to the "yielding" method. This helps our "yielding" methods adhere to the principal of fail-fast.

MSDN Wiki?

Omer van Kloeten — Thu, 21 Aug 2008 08:01:33 GMT

I just love the fact that there's a "Community Content" section to the MSDN documentation. I just found out that there's an undocumented exception coming from the System.Net.Mail.MailMessage.Subject property, I browsed to the relevant MSDN page and simply added a line saying so.

It's a nice copy of the features from PHP's Documentation, but about 10 years too late, and also, mind you, not a Wiki at all. It's comments. In a real Wiki, my corrections to the documentation would be displayed inline with the original documentation. You know, like Wikipedia. Dear Microsoft, please turn it into a real Wiki, at which time your documentation will retain its relevance for me.

On a side note, does anyone know of an add-in for Visual Studio that replaces F1 to a call to Google in my default browser?

Pitfall: Static Field Inline Initialization Order of Execution

Omer van Kloeten — Tue, 19 Aug 2008 14:09:47 GMT

Here's something I fell into today. You have a class that has some members that need to be calculated once. So you use static readonly fields. One of those members is a calculation of some of those members. Take the following code for instance:

public class StaticDemo
{
    static readonly int Sum = A + B;
    static readonly int A = Calculator.GetA();
    static readonly int B = Calculator.GetB();
}

This code will initialize A and B from the Calculator class and initialize Sum from the sum of A and B. However, when you run this code, Sum will equal 0, no matter what A or B are.

Why is this? Apparently, when initializing static fields inline, the order of execution is from top to bottom, regardless of dependencies. Trying to use the same method to initialize an instance field will result in the error A field initializer cannot reference the non-static field, method, or property.

So how do we solve this, you ask?

public class StaticDemo
{
    static readonly int A = Calculator.GetA();
    static readonly int B = Calculator.GetB();
    static readonly int Sum = A + B;
}

Now Sum will really equal the sum of A and B. Simple, but a sure-fire pitfall if you don't know it.

Extension Methods Roundup: Intersect, Union, AsNullable and GroupEvery

Omer van Kloeten — Thu, 31 Jul 2008 21:51:19 GMT

Here we go with the third installment of the Extension Method Roundup. The reason behind these 'code dumps' is that LINQ is a central part of my coding and always find new problems I want to find elegant solutions to. Hope these prove as useful to you as they do to me.

Intersect / Union

Again, shorthand for when you have Enumerable of Enumerable of T and you simply want to intersect or union all of the enumerations in one single call.

public static IEnumerable<T> Intersect<T>(this IEnumerable<IEnumerable<T>> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    IEnumerable<T> returnValue = null;

    foreach (var e in enumeration)
    {
        if (returnValue != null)
            returnValue = e;
        else
            returnValue = returnValue.Intersect(e);
    }

    return returnValue;
}

public static IEnumerable<T> Union<T>(this IEnumerable<IEnumerable<T>> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    IEnumerable<T> returnValue = null;

    foreach (var e in enumeration)
    {
        if (returnValue != null)
            returnValue = e;
        else
            returnValue = returnValue.Union(e);
    }

    return returnValue;
}

AsNullable

I was always missing this method, to coincide with the Cast and OfType methods.

public static IEnumerable<T?> AsNullable<T>(this IEnumerable<T> enumeration)
    where T : struct
{
    return from item in enumeration
           select new Nullable<T>(item);
}

GroupEvery

This takes count items from an enumeration and groups them into a single array.

public static IEnumerable<T[]> GroupEvery<T>(this IEnumerable<T> enumeration, int count)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    if (count <= 0)
        throw new ArgumentOutOfRangeException("count");

    int current = 0;
    T[] array = new T[count];

    foreach (var item in enumeration)
    {
        array[current++] = item;

        if (current == count)
        {
            yield return array;
            current = 0;
            array = new T[count];
        }
    }

    if (current != 0)
    {
        yield return array;
    }
}

I've also gone and updated my LINQ Extensions project on CodePlex with everything I've published since the last update. You're welcome to download and fiddle with it. :)

Asynchronously Preloaded LINQ Queries

Omer van Kloeten — Tue, 29 Jul 2008 20:04:01 GMT

As a rule of thumb, when presented with two independent blocking I/O operations on more than one independent devices, it's best to use threads to create parallel operations, instead of waiting for a single synchronous operation to complete. That way, executing operations O₁, ..., O_n, each of which take T₁, ..., T_n will result in total time where T < T₁ + ... + T_n, instead of T = T₁ + ... + T_n.

Let's take a simple example that consists of the following two operations: The first tries to load all of the files in the My Pictures folder as assemblies (silly), while the other simulates some obscure database operation by calling Thread.Sleep (very silly).

var assemblies = (from file in new DirectoryInfo(@"C:\...\My Pictures").GetFiles("*.*")
                  let assembly = TryLoadAssembly(file.FullName)
                  where assembly != null
                  select assembly).ToArray();

// Do some long database work.
Thread.Sleep(1000);

The code executes synchronously in about two seconds, one second for assembly loading operation and another second holding the thread.

What we could do is enqueue a work item for the ToArray call before the database operation and complete the execution synchronously once we're done. I've coded a short-hand syntax for that:

var assemblies = from file in new DirectoryInfo(@"C:\...\My Pictures").GetFiles("*.*")
                 let assembly = TryLoadAssembly(file.FullName)
                 where assembly != null
                 select assembly;

assemblies = assemblies.Buffered();

// Do some long database work.
Thread.Sleep(1000);

// Force Load
assemblies = assemblies.ToArray();

Once the Buffer call is made, the deferred query begins to run in a separate thread, buffering items into memory, waiting for the query to be executed. Once the query is executed, the buffered items are immediately returned and the iteration completes synchronously. The above code takes approximately one second to run, as both operations run concurrently.

I've attached the code behind this for your consideration and will be adding it to my LINQ Extensions project on CodePlex once I get around to it... :)

public static class ExtensionMethods
{
    /// <summary>
    /// Asynchronously begins buffering an enumeration, even before it is lazy loaded.
    /// </summary>
    public static IEnumerable<T> Buffered<T>(this IEnumerable<T> enumeration)
    {
        // Check to see that enumeration is not null
        if (enumeration == null)
            throw new ArgumentNullException("enumeration");

        return new AsyncEnumerable<T>(enumeration);
    }

    private class AsyncEnumerable<T> : IEnumerable<T>
    {
        private volatile bool shouldContinueBuffering;
        private IEnumerator<T> enumerator;
        private IAsyncResult asyncResult;
        private List<T> buffer;
        private Action bufferAction;
        private object syncLock;

        public AsyncEnumerable(IEnumerable<T> enumeration)
        {
            this.enumerator = enumeration.GetEnumerator();
            this.shouldContinueBuffering = true;
            this.buffer = new List<T>();
            this.syncLock = new object();
            this.bufferAction = this.Buffer;

            this.asyncResult = this.bufferAction.BeginInvoke(null, null);
        }

        private void Buffer()
        {
            lock (this.syncLock)
            {
                // Continue buffering for as long as we can and while there are still items left.
                while (this.shouldContinueBuffering && this.enumerator.MoveNext())
                {
                    buffer.Add(enumerator.Current);
                }
            }
        }

        IEnumerator<T> IEnumerable<T>.GetEnumerator()
        {
            this.shouldContinueBuffering = false;

            // Wait for the last item buffered to finish.
            lock (this.syncLock)
            {
                // End invocation so that exceptions could be throw here.
                this.bufferAction.EndInvoke(this.asyncResult);
            }

            // Iterate over buffered items.
            foreach (var item in buffer)
            {
                yield return item;
            }

            // Continue iterating from the point where we stopped buffering.
            while (enumerator.MoveNext())
            {
                yield return enumerator.Current;
            }
        }

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return ((IEnumerable<T>)this).GetEnumerator();
        }
    }
}

Extension Methods Roundup: Remove, Aggregate, At, AsIndexed and Friends

Omer van Kloeten — Thu, 19 Jun 2008 19:28:00 GMT

Hey hey hey! It's time for another Extension Methods Roundup! Here are some of the extension methods I've written since the last one:

Dictionary's Missing Remove Methods

public static void Remove<TKey, TValue>(this IDictionary<TKey, TValue> dictionary, TValue value)
{
    // Check to see that dictionary is not null
    if (dictionary == null)
        throw new ArgumentNullException("dictionary");

    foreach (var key in (from pair in dictionary
                         where EqualityComparer<TValue>.Default.Equals(value, pair.Value)
                         select pair.Key).ToArray())
    {
        dictionary.Remove(key);
    }
}

public static void RemoveRange<TKey, TValue>(this IDictionary<TKey, TValue> dictionary, IEnumerable<TValue> values)
{
    // Check to see that dictionary is not null
    if (dictionary == null)
        throw new ArgumentNullException("dictionary");

    // Check to see that values is not null
    if (values == null)
        throw new ArgumentNullException("values");

    foreach (var value in values.ToArray())
    {
        ExtensionMethods.Remove(dictionary, value);
    }
}

public static void RemoveRange<TKey, TValue>(this IDictionary<TKey, TValue> dictionary, IEnumerable<TKey> keys)
{
    // Check to see that dictionary is not null
    if (dictionary == null)
        throw new ArgumentNullException("dictionary");

    // Check to see that keys is not null
    if (keys == null)
        throw new ArgumentNullException("keys");

    foreach (var key in keys.ToArray())
    {
        dictionary.Remove(key);
    }
}

String Aggregation

public static string Aggregate(this IEnumerable<string> enumeration, string separator)
{
    return Aggregate(enumeration, str => str, separator);
}

public static string Aggregate<T>(this IEnumerable<T> enumeration, Func<T, string> toString, string separator)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    // Check to see that toString is not null
    if (toString == null)
        throw new ArgumentNullException("toString");

    // Check to see that separator is not null or an empty string
    if (string.IsNullOrEmpty(separator))
        throw new ArgumentNullException("separator");

    return enumeration.Aggregate(string.Empty,
        (accum, item) => string.Format("{0}{1}{2}", accum, separator, toString(item)),
        str => str.Length > separator.Length ? str.Substring(separator.Length) : str);
}

Those are very good for when you want to create strings such as "a, b, c, d".

LastOrDefault

public static T LastOrDefault<T>(this IList<T> list)
{
    // Check to see that list is not null
    if (list == null)
        throw new ArgumentNullException("list");

    if (list.Count == 0)
        return default(T);

    return list[list.Count - 1];
}

This is an optimized version of the original LastOrDefault for lists that allow random access.

At

public static T At<T>(this IEnumerable<T> enumeration, int index)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    return enumeration.Skip(index).First();
}

public static IEnumerable<T> At<T>(this IEnumerable<T> enumeration, params int[] indices)
{
    return At(enumeration, (IEnumerable<int>)indices);
}

public static IEnumerable<T> At<T>(this IEnumerable<T> enumeration, IEnumerable<int> indices)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    // Check to see that indices is not null
    if (indices == null)
        throw new ArgumentNullException("indices");

    int currentIndex = 0;

    foreach (int index in indices.OrderBy(i => i))
    {
        while (currentIndex != index)
        {
            enumeration = enumeration.Skip(1);
            currentIndex++;
        }

        yield return enumeration.First();
    }
}

At provides pseudo-random access to enumerable lists, where needed. I've found use for it in a couple of places which returned indices for non-IList<T> enumerations.

SequenceEqual<T1, T2>

public static bool SequenceEqual<T1, T2>(this IEnumerable<T1> left, IEnumerable<T2> right, Func<T1, T2, bool> comparer)
{
    using (IEnumerator<T1> leftE = left.GetEnumerator())
    {
        using (IEnumerator<T2> rightE = right.GetEnumerator())
        {
            bool leftNext = leftE.MoveNext(), rightNext = rightE.MoveNext();

            while (leftNext && rightNext)
            {
                // If one of the items isn't the same...
                if (!comparer(leftE.Current, rightE.Current))
                    return false;

                leftNext = leftE.MoveNext();
                rightNext = rightE.MoveNext();
            }

            // If left or right is longer
            if (leftNext || rightNext)
                return false;
        }
    }

    return true;
}

This differs from the original SequenceEqual in that it is able to accept two different types of sequences.

AsIndexed

public static IEnumerable<KeyValuePair<int, T>> AsIndexed<T>(this IEnumerable<T> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    int i = 0;

    foreach (var item in enumeration)
    {
        yield return new KeyValuePair<int, T>(i++, item);
    }
}

This is when you need indices, but don't want the overhead of creating an Array<T>.

The Missing SelectMany

public static IEnumerable<T> SelectMany<T>(this IEnumerable<IEnumerable<T>> source)
{
    // Check to see that source is not null
    if (source == null)
        throw new ArgumentNullException("source");

    foreach (var enumeration in source)
    {
        foreach (var item in enumeration)
        {
            yield return item;
        }
    }
}

Oh, come on! Why wasn't there a parameterless SelectMany in the framework? Oh well, here's one.

ToDictionary of IGrouping

public static Dictionary<TKey, IEnumerable<TElement>> ToDictionary<TKey, TElement>(
    this IEnumerable<IGrouping<TKey, TElement>> enumeration)
{
    // Check to see that enumeration is not null
    if (enumeration == null)
        throw new ArgumentNullException("enumeration");

    return enumeration.ToDictionary(item => item.Key, item => item.Cast<TElement>());
}

This is shorthand for when you want to create a dictionary from the result of GroupBy.

Breadth Recursion - a yield Solution to Post's Correspondence Problem

Omer van Kloeten — Fri, 06 Jun 2008 20:07:48 GMT

Post's Correspondence Problem (the other PCP) is a computer science problem, in which you have (and I simplify matters) a set of tiles, each having any number of letters on them from a preset group. For instance, you may have the tiles:

The idea is to create a sequence of tiles (when you can use an infinite amount of tiles from each kind) to get the exact same combination of letters both on the top row and the bottom one. One such combination would be aaaabab, where you would use tiles 4, 4, 2 and 1 to create the sequence: [aa][aa][b][ab] at the top and [a][a][a][abab] at the bottom. If you like, a good mental exercise would be to find the next shortest sequences.

PCP is an undecidable decision problem, which means that no program can be written that could receive a finite set of tiles as its input and return true or false as to whether a combination exists, without the risk of running indefinitely. However, a program that has the risk of running indefinitely that solves the problem can be written: Simply check all sequences of length 1, 2, 3, etc. and stop at the first that is a match.

Writing such a program is a bit cumbersome, as at each sequence of length n, you will have to either save all sequences of length n-1 or recalculate them. Using a classic recursion isn't that useful, since those are used for depth-based analysis, rather than breadth based. Luckily, C# 2.0's yield statement offers us a different type of recursion - the breadth recursion:

IEnumerable<Tile> GetTileSequence(IEnumerable<Tile> tiles)
{
    foreach (Tile tile in tiles)
    {
        yield return new Tile { Top = tile.Top, Bottom = tile.Bottom };
    }

    foreach (Tile sequence in GetTileSequence(tiles))
    {
        foreach (Tile tile in tiles)
        {
            yield return new Tile { Top = sequence.Top + tile.Top, Bottom = sequence.Bottom + tile.Bottom };
        }
    }
}

The above code runs on the set of tiles to create single tile sequences and then uses itself to create sequences one tile longer than itself. It's a bit confusing, I admit, but after a couple of minutes of examining it you may start seeing the hidden beauty of it. Using it to solve the problem would look something like this:

Tile[] tiles = new Tile[] { 
    new Tile { Top = "ab", Bottom = "abab" },
    new Tile { Top = "b", Bottom = "a" },
    new Tile { Top = "aba", Bottom = "b" },
    new Tile { Top = "aa", Bottom = "a" }
};

foreach (Tile sequence in GetTileSequence(tiles))
{
    if (sequence.Top == sequence.Bottom)
    {
        Debug.WriteLine("Match: " + sequence.Top + ", " + sequence.Bottom);
        return sequence;
    }
}

This whole piece of code was written a few days after a little debate I had with @yosit about whether using yield statements to build recursions was a good idea or not. I still hold the firm belief that it usually isn't a good idea, since it's, as you can see for yourself, pretty confusing; That and the fact that most people are used to the classic recursion.

Mechsonomy: Machine Assisted Folksonomy

Omer van Kloeten — Fri, 06 Jun 2008 12:13:02 GMT

Yesterday, in front of the staff and students at my college, I presented my final project for my C.S. B.Sc.. Once I complete it and it gets reviewed this September, I will have completed my duties for the degree.
The project is a research I'm doing for nuconomy and I'll release the code once it's complete. It uses the .NET Framework 3.5 (with C# 3.0) and SQL Server 2005 Integration Services' NLP engine.

The following is the abstract and you can also download the short presentation.

Abstract

The advent of Web 2.0, with its introduction of the concept of user generated data, has posed several problems to those developers aiming to make the navigation in such data as simple as possible.

The problem was commonly met by the coupling of meta-data (tags) to the user-generated content itself, which posed another problem, simply because the vast amounts of data were no match for the small number of website operators to cope with. Thus was introduced the concept of the Folksonomy, or social tagging, which took advantage of the content’s users, asking them to explain what the content was about in an engaging way.

Unfortunately, creating a working folksonomy requires a large and cooperative user base, something that can’t be relied upon.

Automation can be introduced into such communities in order to relieve most of the pressure classic folksonomies place on the user base. By automatically analyzing the user-generated content and meta-content and applying to it a base set of tags, such automation saves users the need to come up with those tags in the first place, leaving only the easier process of correction.

Mechsonomy consists of the following building blocks:

Plain-Text Tagging – user-generated content is taken as-is and processed by a Term Extraction engine to retrieve ‘relevant’ tags.
Markup Analysis – the placement of terms retrieved in the marked-up source is examined, altering terms’ significance.
Web Analysis – the relationship between units of content is examined, altering terms’ significance.
Machine Learning – users interact and rank tags’ relevance to the content, allowing Mechsonomy to learn the impact the site’s markup has on the content’s relevance.

Quick Blurb: Test Tools Toolbar in Visual Studio and Useless Icons

Omer van Kloeten — Mon, 02 Jun 2008 15:41:54 GMT

Let's play a little game. To the left of this text are the five central icons from the Test Tools Toolbar in Visual Studio. Their commands are, in an unordered manner: Test List Editor, Test Runs, Test View, Code Coverage Results and Test Results. Connect the icon to the appropriate command, without checking Visual Studio.

The purpose of this game? To show that whenever you don't think about your icons enough, they're meaningless and therefore useless. There may be a certain logic behind this set of icons, but since it has eluded me for the few seconds I tried to understand it, it's as if it wasn't there in the first place.

LINQ Performance Pitfall - Deferred Execution

Omer van Kloeten — Tue, 27 May 2008 19:37:58 GMT

When using LINQ, queries may bloat up to dozens of lines. My personal style is to take these queries and break them apart to smaller units of logic. To each unit of logic, I append a call to ToArray. @yosit asked me why I did it and I answered I was avoiding a possible pitfall. Here's what I meant.

Take the following code for instance:

static void Main(string[] args)
{
    int[] arr = { 1, 2, 3, 4, 5, 6, 7, 8, 9 };

    var filter = from n in arr
                 where VeryLongOperation(n)
                 select n;

    var cartesian = from n in filter
                    from m in filter
                    select 2 * n + 3 * m;

    var result = cartesian.ToArray();
}

Imagine VeryLongOperation only allows numbers up to 3 and prints the number of times it was called to the debug output. It looks as though the very long operation will run only once per number, so you'll only have three calls, but here's the actual output you get in the debug window once you run this code:

Operation #1
Operation #2
Operation #3
...
Operation #36

This is caused by LINQ's deferred execution, which means that every time an item is taken from any of the loops, it will go back to the first filter and call the very long operation again. This means that you have 3 calls that cause 9 inner loops (27 calls together) and 6 that don't cause inner loop calls. 3 + 27 + 6 = 36.

Let's make a slight alteration to the original filter query:

var filter = (from n in arr
              where VeryLongOperation(n)
              select n).ToArray();

This forces LINQ to execute the query now. Sure, there's a slight overhead of creating a new array, but it's a static array, so that mitigates the problem a bit. Now the debug window looks like this:

Operation #1
...
Operation #9

This is a neat trick and is one of the first things that I look for when reviewing code with multiple LINQ statements.

SQL Server 2005 Analysis Services's ADOMD.NET Connection Pooling, or Lack Thereof

Omer van Kloeten — Tue, 27 May 2008 08:33:30 GMT

First of all, if you've come here looking for how to activate connection pooling when using SSAS 2005 via ADOMD.NET, you're in for a little surprise - there is none by design.

Loading a new connection can take up a long time, since with every new session to the database, all of the metadata and security context has to get loaded too. I've decided to perform some mental gymnastics and try and implement a simple connection pooling mechanism. Here is the (very) basic version of the mechanism, which I may continue to post changes to as they accumulate:

public static class AdomdConnectionPool
{
    private static Dictionary<string, Dictionary<string, ArrayList>> pool = 
        new Dictionary<string, Dictionary<string, ArrayList>>();

    /// <summary>
    /// Gets a connection from the pool or creates one if one does not exist.
    /// </summary>
    public static PooledConnection GetConnection(string connectionString)
    {
        // Pooling (See the Poll method)
        ValidateListExistance(connectionString);
        return GetConnectionFromPool(connectionString);
    }

    private static PooledConnection GetConnectionFromPool(string connectionString)
    {
        KeyValuePair<string, ArrayList> session =
            new KeyValuePair<string, ArrayList>(null, null);

        lock (pool[connectionString])
        {
            if (pool[connectionString].Count > 0)
            {
                // Available session exists. Use it.
                session = pool[connectionString].First();
                pool[connectionString].Remove(session.Key);
            }
        }

        // No available connections exist. Create a new one.
        if (session.Key == null)
            return CreateNewConnection(connectionString);

        // A session exists
        AdomdConnection pooledConnection = new AdomdConnection(connectionString);
        pooledConnection.SessionID = session.Key;

        try
        {
            pooledConnection.Open();

            // Register the session with the pool.
            PooledConnection poolItem = new PooledConnection(pooledConnection, session.Value);
            poolItem.Disposed += new Action<PooledConnection, EventArgs>(poolItem_Disposed);

            return poolItem;
        }
        catch (Exception ex)
        {
            // Connection probably expired. Try again.
            return GetConnectionFromPool(connectionString);
        }
    }

    private static void ValidateListExistance(string connectionString)
    {
        lock (pool)
        {
            if (!pool.ContainsKey(connectionString))
                pool.Add(connectionString, new Dictionary<string, ArrayList>());
        }
    }

    private static PooledConnection CreateNewConnection(string connectionString)
    {
        // Create a new connection and register it with the pool.
        PooledConnection poolItem = new PooledConnection(
            new AdomdConnection(connectionString),
            new ArrayList());
        poolItem.Disposed += new Action<PooledConnection, EventArgs>(poolItem_Disposed);

        poolItem.Connection.Open();

        return poolItem;
    }

    private static void poolItem_Disposed(PooledConnection sender, EventArgs e)
    {
        Dictionary<string, ArrayList> connections = pool[sender.Connection.ConnectionString];

        try
        {
            // Close the connection, but keep the session alive.
            sender.Connection.Close(false);

            lock (connections)
            {
                // Reclaim the connection to the pool.
                connections.Add(sender.Connection.SessionID, sender.ExtraData);
            }
        }
        catch
        {
            // Can't close connection? Don't let it back in the pool.
            // We don't really care why, though. If necessary in the future, log.
        }
    }
}

Each new connection has a SessionID property, which is unique and is given to each connection that is opened without an existing value in the SessionID property. If there is a value in the property before the connection opens, the connection connects to that session once Open is called. It may have been wiser to use the same SessionID for all connections, but since timeouts may happen on old connections (see the end of the poolItem_Disposed method), I decided to use many, fresh connections. I admit I have no idea whether it's a best practice, but I have yet to see evidence otherwise.

Each pooled connection has two properties we need to use: Connection and ExtraData:

public class PooledConnection : IDisposable
{
    /// <summary>
    /// Creates a new instance of <see cref="PooledConnection" />.
    /// </summary>
    internal PooledConnection(AdomdConnection connection, ArrayList extraData)
    {
        this.Connection = connection;
        this.ExtraData = extraData;
    }

    public AdomdConnection Connection { get; private set; }
    public ArrayList ExtraData { get; private set; }

    void IDisposable.Dispose()
    {
        if (this.Disposed != null)
            this.Disposed(this, EventArgs.Empty);
    }

    internal event Action<PooledConnection, EventArgs> Disposed;
}

The ExtraData property is there to store connection-specific data, such as a list of session-scoped members already created at runtime on this session, etc.

Using the mechanism is very simple and resembles using a simple ADO.NET connection:

using (PooledConnection pooledConnection = AdomdConnectionPool.GetConnection(connectionString))
{
    // Execute the query
    AdomdDataAdapter adapter = new AdomdDataAdapter(query, pooledConnection.Connection);
}

Please note that you can not use this sort of connection pooling when you're using the integrated role-based security, unless you save your sessions according to roles. This has not and will not be implemented in my connection pooling mechanism and if you need it, you'll have to write one for yourself.

I would love to hear comments about the whole thing. Remember that it's pretty basic, but I would love to make it more interesting and intricate and most of all - useful.