Omer van Kloeten's .NET Zen

Programming is life, the rest is mere details

News

Note: This blog has moved to omervk.wordpress.com.

Omer van Kloeten's Facebook profile

Omer has been professionally developing applications over the past 8 years, both at the IDF’s IT corps and later at the Sela Technology Center, but has had the programming bug ever since he can remember himself.
As a senior developer at NuConomy, a leading web analytics and advertising startup, he leads a wide range of technologies for its flagship products.

Get Firefox


powered by Dapper 

.NET Resources

Articles :: CodeDom

Articles :: nGineer

Culture

Projects

This Blog Has Moved

Please find its new home at omervk.wordpress.com.

The blog has been made read-only, but will not be deleted for now. All of the content has been replicated to the new blog, so you can continue the discussion on the older posts there.

I would like to thank weblogs.asp.net for being my home these past few years.

Omer

The Anti-Social Network

(I’ve returned from my trip a few months ago, but didn’t have enough to blog about; hopefully, this is my returning to a semi-regular posting schedule; also, I’ve decided to veer off .NET in some of my posts, focusing on more issues that I like to talk about and offer my opinions on)

(Update: This article is now featured in Hebrew in Gal Mor’s blog “Holes in the Net”)

CC-BY-NC, Min Jung Kim Social networks have been flourishing in the past few years, and have let us reach more people than we could before them. I, personally, use both Facebook and Twitter on a daily basis. The trend of ‘socialized’ applications is steadily marching on. A simple example of that is e-commerce sites looking to increase their conversion rate with you using recommendations from your friends.

However, there is yet another trend coming up that we should worry about, which is users trying to force anti-social behavior. Here are a couple of examples:

  1. Facebook introduced the ‘Like’ feature in February 2009, making it easier for users to say that they like a certain something: a post, a picture, a link, etc. This feature was immediately put to good use, with people ‘liking’ things, instead of creating long comment chains. ‘Liking’ something was much easier, too, being a single-click action, and increased the amount of participation. The Like feature was a good thing.

    Lately, voices have come up in favor of creating a ‘Dislike’ button. Users feel that since they have an easy way to notify their friend that they like his or her post, that they should equally be able to notify them of not liking it, discouraging the person from continuing this kind of activity.

    To use the ‘Dislike’ feature would be like saying to someone’s face that their joke was not funny or that they are ugly. Face to face, you might only do that to your closest friends and family and even then in jest, but online it’s much easier to be anti-social. The socially acceptable alternative to the ‘Dislike’ button would be to ‘Hide’ that friend’s updates, which unlike ignoring someone in the real world, is something the friend will never know about and will therefore never get offended by.
  2. The way to use Twitter is not something users can agree on. One thing that is widely agreed upon, though, is that following someone means you care what they say. That leads most people to the mistaken belief that the more followers you acquire, the more people will believe you are important and influential.

    One of the ways in which users are attempting to inflate their follower count is the contemptible habit of expecting auto-follow-back. This means that the user follows you, waits for you to follow them back and then un-follows you if you haven’t. This widely embraced practice causes unsuspecting users to be barraged with a very large number of follow notifications from people, brands and companies they have no interest in.

    This is downright adamant to spam, which we can all agree is one of the least socially acceptable activities you can partake in online.

These examples just go to show the trend of users wanting to add anti-social behavior to a social platform.

Social networks, on their part, should never allow themselves to be catalysts for anti-social behavior. Anti-social activities, such as hiding users or reporting them or their actions should remain hidden from all other users and must remain in confidence between the user and the platform. The real world parallel to this would be ‘telling on’ someone, which is frowned upon.
Facebook tries to avoid that by making all positive social activities more open, while keeping the anti-social activities on a one-on-one basis between the user and the platform. Unfavorable comments are still prevalent, but are unavoidable.
Twitter is, on the other hand, lagging far behind by keeping their network very much bare of personal filters (no way to hide unwanted tweets) and allowing users to easily find out who un-followed them, to name just a few issues.

The rule-of-thumb I present here does not stop with social networks, but continues to other socially-influenced platforms. Socially-enabled games follow this rule too: FarmVille, which at the time of writing of this post is the most popular and fastest growing game of all time on Facebook, does the same.
Players have a multitude of socially-positive activities to choose from, such as sending each other gifts, taking in animals their friends put up for adoption, helping out on friends’ farms and even racing each other to the top of their local hall of fame in the long run. On the other hand, there are no socially-negative activities in the game: players can not steal crops or animals from friends, mess up their farms, etc.

Social platforms should make it easy to broadcast socially-positive activities and hard to broadcast the negative ones. Once they follow this simple frame of mind, they will reap the psychological and sociological benefits.

Hiatus

In 15 days’ time, I’ll be taking off on a four months long trip around the world. I’ll be hiking and travelling my way through Japan (March-May), the United States of America (May-July) and the Netherlands (July). I’ve been planning this trip for a long time now, since I came back from Japan for the first time in August 2007 (the photo on the right was taken near the Tsurugaoka Hachiman-gu Shrine in Kamakura), and have taken a long leave from work to accomplish it. What this means is that from March 10th you can expect a long pause in my semi-regular updates.

I am not dead :)

If you’re a reader who lives along my path and want to meet up for a beer and a lively chat, ping me via the contact form. I’ll be reading my emails every once in a while, so don’t worry if you only get an answer after a few days.

If you want to read about my trip, I’ll be posting regular updates to my new travel blog.

Circumventing the KB957543 .NET 3.5 SP1 Regression Bug

A couple of days ago I hit a regression bug in .NET 3.5 SP1, in which when you have a generic class that implements ISerializable and has static variables – you can not serialize it using a BinaryFormatter without your application either hanging (x86) or raising an exception (x64 – a TargetInvocationException containing an OutOfMemoryException). This only happens if you use a reference type as a generic argument.

It’s already well known, but I have yet to find a workaround documented anywhere. You could simply install the hotfix, but well, I wouldn’t if I were you – it hasn’t been thoroughly tested yet. Moreover, you might not even be able to do so due to either internal politics, strict IT rules or the fact that you simply do not have control over the hosting server.

Let’s take the simplest class that causes the issue:

[Serializable]
public class MyClass<T> : ISerializable
{
    private static int list = 0;

    public MyClass()
    {
    }

    protected MyClass(SerializationInfo info, StreamingContext context)
    {
    }

    void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
    {
    }
}

When using the class as such:

using (MemoryStream stream = new MemoryStream())
{
    BinaryFormatter formatter = new BinaryFormatter();
    formatter.Serialize(stream, new MyClass<string>());
    stream.Position = 0;
    MyClass<string> item = (MyClass<string>)formatter.Deserialize(stream);
}

The last line will hit the bug.

To work around this issue, simply move your static variables into a new subclass:

[Serializable]
public class MyClass<T> : ISerializable
{
    private static class KB957543
    {
        public static int list = 0;
    }

    public MyClass()
    {
    }

    protected MyClass(SerializationInfo info, StreamingContext context)
    {
    }

    void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
    {
    }
}

You can still access all of your static variables and you don’t hit the bug.

Note that when you use anonymous methods or lambdas, they are cached as static variables of the type, meaning that you will have to manually type all of your lambdas.

Here’s an example of such a type that is prone to the bug:

[Serializable]
public class MyClass<T> : ISerializable
{
    public MyClass()
    {
    }

    public static string Concat(IEnumerable<int> numbers)
    {
        return string.Join(", ", numbers.Select(i => i.ToString()).ToArray());
    }

    protected MyClass(SerializationInfo info, StreamingContext context)
    {
    }

    void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
    {
    }
}

If we look through Reflector, we can see that there is a cached delegate in our type:

Since this is a static member, it makes the type susceptible to the bug and we now need to manually create the cached member ourselves:

[Serializable]
public class MyClass<T> : ISerializable
{
    private static class KB957543
    {
        public static readonly Func<int, string> ToString = i => i.ToString();
    }

    public MyClass()
    {
    }

    public static string Concat(IEnumerable<int> numbers)
    {
        return string.Join(", ", numbers.Select(KB957543.ToString).ToArray());
    }

    protected MyClass(SerializationInfo info, StreamingContext context)
    {
    }

    void ISerializable.GetObjectData(SerializationInfo info, StreamingContext context)
    {
    }
}
Let SQL Server Tell You Which Indexes to Rebuild

When index fragmentation becomes too high, indexes will be very inefficient. Other than planning a good index design, you should rebuild / reorganize your indexes every once in a while.

SELECT 'ALTER INDEX [' + ix.name + '] ON [' + s.name + '].[' + t.name + '] ' +
       CASE WHEN ps.avg_fragmentation_in_percent > 40 THEN 'REBUILD' ELSE 'REORGANIZE' END +
       CASE WHEN pc.partition_count > 1 THEN ' PARTITION = ' + cast(ps.partition_number as nvarchar(max)) ELSE '' END
FROM   sys.indexes AS ix INNER JOIN sys.tables t
           ON t.object_id = ix.object_id
       INNER JOIN sys.schemas s
           ON t.schema_id = s.schema_id
       INNER JOIN (SELECT object_id, index_id, avg_fragmentation_in_percent, partition_number
                   FROM sys.dm_db_index_physical_stats (DB_ID(), NULL, NULL, NULL, NULL)) ps
           ON t.object_id = ps.object_id AND ix.index_id = ps.index_id
       INNER JOIN (SELECT object_id, index_id, COUNT(DISTINCT partition_number) AS partition_count
                   FROM sys.partitions
                   GROUP BY object_id, index_id) pc
           ON t.object_id = pc.object_id AND ix.index_id = pc.index_id
WHERE  ps.avg_fragmentation_in_percent > 10 AND
       ix.name IS NOT NULL

The above query will give you a list of recommended index rebuild / reorganize statements for your database, according Pinal Dave’s 10-40 rule, although you are welcome to tweak it to your liking. It supports non-partitioned as well as partitioned indexes. If you want a more intense check for fragmentation, change the last NULL in the dm_db_index_physical_stats call to 'SAMPLED' or even 'DETAILED' (include quotes).

It’s a handy little tool for database administrators and saves a lot of the hassle of monitoring index fragmentation.

Update: Added multi-schema support as suggested by MJ12 and another check for null index names.

SQL Server Management Studio 2008 IntelliSense Doesn't Recognize Special Characters

ssms

I just filed a new bug with Microsoft Connect. I certainly hope this one doesn’t get shrugged off like many of my other bugs did.

Want to reproduce it yourselves? Just create a table with a character that can only be valid inside the context of brackets (like a comma or braces) and then try to select from it. Don’t keep the CREATE clauses in the same query window or it might just work. See the screenshot below.

This is very frustrating because this means that not only is there no IntelliSense for these objects, but IntelliSense now gets in the way of actual querying as it will autocomplete incorrect names.

Adobe AIR and Hebrew Fonts

Twhirl(I’ve noticed quite a few people were having this problem, so I decided to blog about it. This might have to do with all non-English fonts, but I experienced it only with Hebrew)

TweetDeck

The only reason I use Adobe AIR is for the Twitter application Twhirl. However, I noticed the Hebrew text was incorrectly displayed (word order was reversed).

I did not find a solution for this online and the guys at Twhirl didn’t know what to make of this.

After toying with a few of the options, I finally found the answer – the fonts used by Twhirl were in Hebrew, but AIR wasn’t playing nicely with them. I switched from Calibri to Tahoma and found that the text was just fine.

Both @effifuks and @JonathanRauch (with TweetDeck, where he didn’t see Hebrew text at all – see the left screenshot) experienced the same issue.

MSBuild Script to Compress All JavaScript Files in a Project
I’ve got one project in my solution which has a lot of JavaScript files and they keep on coming. We’ve been using the YUI Compressor for quite a while and it’s proven an effective tool. After a lot of time of fiddling with the project’s MSBuild script, I came up with the following:
<Target Name="BeforeBuild">
  <MakeDir
Directories="compressed\%(Content.RelativeDir)"
Condition="(%(Content.Extension) == '.js') And (!(Exists('compressed\%(Content.RelativeDir)')))" /> <Exec
Command="java -jar yuicompressor-x.y.z.jar --type js -o compressed\%(Content.Identity) %(Content.Identity)"
Condition="%(Content.Extension) == '.js'" /> <CreateItem
Include="compressed\%(Content.Identity)"
Condition="%(Content.Extension) == '.js'" /> </Target>

The above takes all of the JavaScript files in your project, compresses them (in the same relative directories) into the compressed directory and then adds them to the project, in case it gets deployed anywhere.

Note that I’m using the Extension, Identity and RelativeDir well-known item metadata attributes in order to impose batching, since batches causes loops instead of the string concatenation that happens when you reference the items themselves.

Your Mouth Says Windows-1255, But Your Eyes Say ISO-8859-1

I recently wrote an engine that gets XML files stored at our clients’ servers using HTTP requests. One of our clients decided to serve the XML file with one encoding and encode the file itself with another. This posed a problem to XDocument.

The client decided to encode their XML using the Windows-1255 encoding (Hebrew), noting the encoding correctly in the XML’s declaration, but served the file stating the ISO-8859-1 (Latin) encoding. This meant that I couldn’t just use XDocument’s normal Load method to load directly from the stream because XDocument looks at the HTTP headers and takes the document’s encoding from them.

Here’s a snippet of the code I used to get over that:

using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
    // Use response's charset.
    var encoding = Encoding.GetEncoding("ISO-8859-1");

    if (!string.IsNullOrEmpty(response.CharacterSet))
        encoding = Encoding.GetEncoding(response.CharacterSet);

    byte[] bytes = ReadStream(response.GetResponseStream());

    // Get the XML with the response's charset.
    string xml = new string(encoding.GetChars(bytes));
    int endOfDeclaration = xml.IndexOf("?>");

    if (endOfDeclaration != -1)
    {
        // Try to find out the encoding from the declaration.
        string decl = xml.Substring(0, endOfDeclaration + 2) + "<duperoot />";
        XDocument declDoc = XDocument.Parse(decl);
        var docEncoding = Encoding.GetEncoding(declDoc.Declaration.Encoding);

        if (docEncoding == encoding)
            return xml;
        else
            return new string(docEncoding.GetChars(bytes));
    }
    else
    {
        // Not XML or something... Send up.
    }
}

What I did here was to create a new document with the original XML’s declaration (the Latin characters which make up the XML’s declaration always have the same byte position), add a dupe root and parse that to get the name of the encoding used by the document. I then use that encoding to decode the document correctly.

Note that I’m using ISO-8859-1 as the default response’s encoding, since that is what HTTP’s specification demands.

HTML to JavaScript HTML DOM Converter

One of our products demands converting quite a large amount of HTML to equivalent scripts, written in JavaScript that create said HTML. Looking around the Internet, I found no tool that will automate this process for me, so I went ahead and created this simple little application.

htmltojsdom

I've written it using HTML Agility Pack and Simple CSS Parser. It’s hardly perfect and might incorrectly reference attributes, but I’ve tweaked it long enough for it to work, I’m guessing, 95% of the time.

The source is not yet included (I want to set it up as a project on CodePlex later on), but for now you can download it from here.

More Posts Next page »