Glavs Blog

Getting better performance in web apps with a few lines of script

Glav — Sat, 27 Jun 2009 09:53:45 GMT

We had a situation where a web application was utilising near 100% of CPU all the time under light load and it was proving very hard to reduce this significantly through code optimisation. We were performance testing and profiling, but only seeing marginal gains.

The infrastructure that was hosting the web application was as follows:

Virtual Server (Hosted via VMWare Server) running Windows Server 2003 64 bit
.Net 3.5
Dual Virtual CPUs
2 Gb of memory.

Now I am not a big fan of virtualisation for hosting high performance web sites but we were stuck with this. At any rate, we had consistently high CPU utilisation for seemingly no real reason. Some of the most simple requests were generating high CPU usage.

After some investigation, and also the help of a friend and very smart dude, Scott Forsyth (who works for Orcsweb a high profile web hosting company) we decided to try setting Internet Information Services to 32 bit mode to see if any difference was seen.

To cut a long story short, we changed Internet Information Services to run in 32 bit mode. This simple switch, yielded significant lower CPU utilisation and in some case, almost cut the CPU utilisation in half!

Why would this be so? Well, it turns out there are a few reasons for this. Probably the primary reason is the fact the we get less information within the level 1 cache of the processor because of the increased structure size (pointers are bigger) within 64 bit systems. The same information in a 64 bit system occupies more memory than its 32 bit counterpart, therefore we can fit less into the processor cache. More information on this can be found here.

Its really easy to change IIS to 32 bit mode. The steps are:

To enable 32 bit mode on IIS in 64 bit windows, at a command prompt type:
cscript %SYSTEMDRIVE%\inetpub\adminscripts\adsutil.vbs SET W3SVC/AppPools/Enable32bitAppOnWin64 1
Once this is done, 32 bit mode is enabled but we need to ensure that the 32 bit DLL’s are used as IIS will still have the 64 bit DLL’s defined for the ISAPI handler. To do this, all we have to do is re-register ASP.NET. At a command prompt, simply type:
%SYSTEMROOT%\Microsoft.NET\Framework\v2.0.50727\aspnet_regiis.exe –i
Finally, just make sure the status ASP.NET version 2.0.50727 (32-bit) in the web services extension, within IIS, is set to allowed.
(More information on this process can be found here http://support.microsoft.com/kb/894435)

And that’s it. A few lines of script, and we had effectively reduced CPU utilisation by somewhere in between 30% and 50%!

Conclusion

So my rule is this, If you are:

Running Windows server in 64 bit mode.
Running a standard .Net web application
Do not need a large memory address space (ie. over 4 gigabytes)
Have no special 64 bit optimisations

Then this change will yield immediate performance improvements in your application, and in some cases (dependent on what your application does), yield significant performance improvements.

FromTheGrey – Arts Site

Glav — Sun, 10 May 2009 02:08:56 GMT

Recently, I have been working on creating an arts based website called “FromTheGrey” at http://FromTheGrey.com

Its a site devoted to showcasing the pictures and artwork created by my wife Michele. Previously, Michele has been very reluctant to showcase these pieces, but I really like them, as do many others I know, so it took some time and coaxing to convince Michele to show them to the world.

However, I finally managed to do. As a result, “FromTheGrey” was born. Its a relatively simple site but I hope you like the art on the site and if so, would really appreciate you recommending it to others.

Its currently simple but functional, and will have many more pieces of work from Michele on there, as well as enhanced styling and functionality.

For now though, we would love your feedback which can be submitted via the Contact Form on the site.

For the technically inclined, it utilises ASP.NET 3.5 Webforms, jQuery 1.3.2, and jQuery plugins for the dialog box and gallery effects, Thickbox and Galleria respectively.

AjaxView – Javascript profiling

Glav — Sun, 10 May 2009 00:25:36 GMT

A little while ago, I posted about a tool called AjaxView to allow you to profile javascript code within your web applications.

Well, I was contacted by Adam who is working on this tool about its latest incarnation. It appears had a lot of work done to it, and now looks quite slick and is really nice to use. Previously, it was quite raw in use and also with the metrics its provides. Now, it acts very similar to one of my favourite profilers, Ants Profiler.

What does it do?

This tool injects profiling code into your javascript that enables it to analyse your javascript code and provide accurate metrics around all the operations being performed, including anonymous functions. It then provides these metrics to the Visual Studio performance tools for you to view in a very simple and familiar manner.

How do you use it?

You need to be using IIS7 in Integrated mode for this tool to work. A full set of install instructions is listed here along with the installation files. It installs pretty easily.

What do you get?

In short, you get detailed metrics on each page and javascript file in your site. When access a site with the AjaxView profile tools installed, all metric information is automatically being recorded. You can then access a web site on your local machine that is installed when AjaxView is installed. (Something like http://localhost/AjaxView).

When you do that, you can view the session profiles you have created (or the default one that is created for you on install). Something like this:

Select any of the files on the right which constitute all the files access by your web app (either a page or javascript file). Select the ‘Get profile data’ button and it launches Visual Studio and provides the profile metric data so that you can view it within Visual Studio. You end up getting access to detailed multiple views like the ones shown below:

All in all, a pretty useful tool to look at your Javascript performance. I recommend you take a look. Get it from here.

Regaining results from a performance test run

Glav — Wed, 29 Apr 2009 12:48:33 GMT

I have been doing a lot of performance testing lately, and one of the worst things that can happen during a performance run, is when you don’t get any results. Performance testing is expensive, from a time and resources perspective, so getting results from a run is paramount.

In my scenario, I have everything automated to kick off a performance run, at 7.00pm every night, run for 12 hours, then terminate. I collect metrics via Visual Studio Team Suite which is used to drive my performance tests, but I also have automated the collection of performance metrics on all related servers (web, sql etc…)

On one of my recent runs, I came into the office in the morning, and checked on the test rig. The test appeared to be still running, even well beyond the termination time. VSTS reported that it was trying to abort the test, but things were still running. Some of the test agents refused to respond.

I pressed the stop button to stop the test in VSTS and while I was waiting for that, loaded up the test results so far. The business were eagerly awaiting the results of the tests and it looked promising so far.

Visual Studio began to refuse to respond after a while and was continuing to try and stop the test. After a long wait, I simply killed off the devenv.exe process, and restarted the Visual Studio Test controller service.

I then went to load my test data from the run. Oh dear, the test run that I was looking at not more than half an hour ago, did not appear. Nothing listed on the ‘Completed Tests’ tab, and nothing within the ‘Manage Test Results’ option either.

Crap. A business person walks by and asks how the test run went. “Still analysing the data” was my response. Double crap. What to do.

After looking at the SQL database where the load results are stored, it was apparent there was a bunch of data in there, that simply was not showing. Looking through the raw data did not appeal to me very much.

I had a look in the ‘LoadTest’ table in the LoadTest database I was using to see if there was any data.

I noticed there was no ‘End Time’ defined for my run. It was simply set to NULL

. The Start Date was correct, so I simply copied the StartTime to the EndTime column, changed the date to the next day, and set the time to AM instead of PM (thus setting my 12 hour run).

And now I have normal access to all my test results. Whew…. crisis averted.

Hope this helps someone.

Animated SiteMap Ajax Control

Glav — Thu, 12 Feb 2009 11:18:53 GMT

Today, I finally released on Codeplex a control that I had been working on over a year ago, called the Animated SIteMap Control. Its an AJAX Control toolkit control and it has been prototype form for a very long time now.

I had planned on releasing it into the AJAX Control toolkit but I got sidetracked and haven’t touched this in over a year. It works fine and probably needs a little refinement but has configurable animation speeds, full support for CSS styling and various other options.

Basically, it creates an ‘exploding’ like circular animation of the nodes defined in your web.sitemap file. You can drill down into nodes that have subnodes, or step back from the selected node, and the nodes of the same level are all animated in a circular fashion around the main or home node.

You can view a very small and brief example of this here: http://www.theglavs.com/ControlTestPage.aspx. Feel free to download it and use it if you like it. Hopefully, you may get the time to contribute to it and make it better. I know my time is short and I just didn’t want to waste it. Perhaps even a jQuery port might come of it.

At any rate, if you end up using it, changing it or whatever, I would love to know. Hope you like it.

EOAST #4 – Evolution of a software thingy Part 4

Glav — Mon, 26 Jan 2009 01:45:00 GMT

So its been ages since my last post on this pet project of mine. Its time to get down to the nitty gritty on this and see if it is actually working the way I want it to. Hopefully, you have read parts one, two and three in previous posts to get a good idea on what I am trying to achieve.

Basically, its a collect, store and display mechanism to store almost any information in a SQL database. You only need concentrate on implementing collection logic, or only display logic or both if you so choose. Its up to you. The storage and querying of this information is done for you, as is the scheduling of collection. Additionally, its gotta be real robust. If someone writes a crappy information provider, the system should not die. Also, I want to be able to map an information provider to one or more display providers, all easily.

I have dubbed this the InformationAggregator.

So I am happy to say its actually working reasonably well at the moment. Since its a pet project of mine, the fact it is actually up to a reasonable level of functionality is quite an achievement. Normally I kinda let things go once I have satisfied my initial curiosity, or whatever it was that got me interested in doing it. What I haven’t really spent much time with is the user interface. Its really just a set of buttons on a form to exercise the API.

At any rate, the basic details are as follows.

There are 2 main assemblies: InformationAggregator.Core.dll and Information.Aggregator.Engine.dll.

If you want to implement an information collection provider or an information display module, you only need to reference the Core assembly. The Engine assembly contains all the functionality to start the hosting and collection engine.

Note: Since this codebase is always being revised, its prone to change so there maybe much refactoring in the future. In fact, part of it will be happening after this post ":-)

Writing an Information Collection provider

Say you want to write a module which collects email, accepts user input, reads a users twitter entries, or some other information collection mechanism. You only need to Implement 2 interfaces: IPluginModule, & IInformationProvider.

IPluginModule has only 1 method,

QueryPluginResponse QueryPluginModule()

which acts to provide some metadata around itself. Here is an example:

public QueryPluginResponse QueryPluginModule()
{
    QueryPluginResponse response = new QueryPluginResponse();
    response.MajorVersion = 1;
    response.MinorVersion = 0;
    response.PluginDescription = "Test Manual Collection. Only sends information to engine once collected from keyboard entry";
    response.PluginName = "ManualCollector";
    response.Publisher = "Glav";
    response.PublisherLink = "http://www.theglavs.com";
    response.RevisionNumber = 0;
    response.SupportsCollectionNotification = false;
    response.TypeOfService = ServiceType.InformationProvider;

return response;
}

IInformationProvider also requires the implementation of only 1 method,

List<GenericInformationItem> GetAllInformation()

Here is an example:

public List<GenericInformationItem> GetAllInformation()
{
List<GenericInformationItem> list = new List<GenericInformationItem>();

    Random rnd = new Random(DateTime.Now.Millisecond);
    for (int i = 0; i < 3; i++)
    {
        GenericInformationItem item = new GenericInformationItem();
        item.Author = "Paul Glavich";

        item.BodyTextData = string.Format("This is dummy body entry ID:# {0}. The current date is {1} and the time is {2}", rnd.Next(1,200),DateTime.Now.ToLongDateString(), DateTime.Now.ToLongTimeString());
        item.InformationSource = "Unit Test";
        item.ReceivedDateTime = DateTime.Now;
        item.Originator = "TestInformationProvider";
        item.PublishDate = DateTime.Now;
        item.SourceProvider = PROVIDER_NAME;
        item.Summary = "Just a test item";

        item.Title = string.Format("Test Item #{0}", i);
        list.Add(item);
    }

return list;
}

This implementation simply sends 3 items of information to the InformationAggregator engine when its asked to do so. This particular implementation is well suited to periodic, or scheduled collection. The engine can be asked to collect information from modules every minute, hour, day, week etc.

The engine also supports manual collection where it does not schedule collection at specific intervals, but waits for the provider to send it information. A third option is where the engine instructs the module to begin collection, then returns to its business, allowing that module to take its sweet time about collection (it may be very time consuming) and then send it the collected items. Status events of the collection process is also supported.

Note that the implementation is very simple. It would be easy to write some logic to query some RSS feeds, twitter feeds, or whatever. All the scheduling and storage is performed via the engine through configuration. Currently I have a very crude user interface (and a I mean very crude) to control all this. Basically some buttons to exercise the functionality of the API:

Only collection modules that have been registered, have their items accepted and stored via the engine in the database. When registering, the IQueryModule interface is invoked to gather and display some metadata to display for confirmation when registering:

Writing an Information Display provider

To display information, again we implement only 2 interfaces: IPluginModule, & IInformationDiplay.

IPluginModule was already discussed above and is a pre-requisite for any module, either display or collection.

IInformationDiplay has 2 methods to implement:

void PushInformation(List<GenericInformationItem> infoItems );

void PushStatus(QueryPluginResponse plugin, int percentCollectionCompleted, CollectionStatus statusIndicator, string statusMessage);

The ‘PushInformation’ method has the information ‘pushed’ to it, and you can do what you want with the info from there. (Note: Not really happy with these method names, but I’ll leave them for now)

The ‘PushStatus’ method gets called repeatedly during the collection phase, if the information provider is providing this information.

The display provider can be a simply console app, or more typically, a WPF application. These all need to be registered as described above.

However, one more pre-requisite is required if its a WPF application. A parameterised constructor must be provided that accepts the WPF window to be used for display. The engine will search the WPF application for this constructor and construct an instance of this window when creating this module, and maintain this, so that display items can be sent to the module, and threading affinity can be easily achieved.

Linking providers to display modules.

So we can easily create an information collection provider. We can also easily create a display provider. How do we get information from collection provider to display provider. We simply map one or more collection modules to a display provider. Here is an example of the (again very crude) UI to do this:

This UI shows 2 information providers, ManualCollector and TestInformationProvider. Currently, only the WPFDisplayProvider#2 is mapped to the TestInformationProvider, with the BalloonPopup display provider available, but not mapped to this module. This means that any information that the TestInformationProvider collects, is currently automatically sent to the WPFDisplayProvider#2. If we want that information to be sent to 2 display providers, we can simply click on the BalloonPopup display provider and move it into the mapped display providers.

Now when the engine starts, all information collected by the TestInformationProvider is automatically sent to both the WPFDisplayProvider#2 and the BalloonPopup display provider. Basically you can pick and choose how your information is collected, and how it is displayed.

So thats the concept. Control the collection of your information, and choose a display method that matches what you want. Additionally, have the information stored in an easily queryable store.

There is actually much more, but I’ll leave it for now. Next step will be a demo, so look for a short video on how it works and specifically the WPF implementation of the display providers.

Some interesting links for ASP.NET

Glav — Fri, 09 Jan 2009 06:29:51 GMT

Its been a while since I have done any technical stuff as I have been on holidays, relaxing, and generally a lot of nothing. Certainly nothing technical related and have really enjoyed spending time with friends and family.

Over this period, Wally has produced some good material around ASP.NET which I feel obligated to share with you….. (ok, so maybe Wally coerced me a bit….)

Firstly, Wally has done a new ASP.NET Podcast:

ASP.NET Podcast Show #134 - General Thoughts on Windows Azure - Audio only

Some good discussion around Cloud computing and Windows Azure in there.

Also, Wally has produced an e-Book on some of the ASP.NET 3.5 SP1 features and its located here.

Go check em out. Make Wally a happy Wally.

New ASP.NET Podcast on jQuery support

Glav — Sat, 06 Dec 2008 05:08:33 GMT

Wally has done another good podcast which can be found here on jQuery support in Visual Studio 2008 and how it compares to ASP.NET AJAX by way of some simple examples.

jQuery is a great AJAX library and well worth investing some time in getting familiar with. In fact, you can start right now by listening to the podcast :-)

Debugging Silverlight apps using WinDbg

Glav — Wed, 03 Dec 2008 00:29:12 GMT

While working on a silverlight application for my current engagement, we were seeing some weird memory issues and application crashes. I won't bore you with the initial investigation of the issue, but it turns out we need to delve deep into what was going on behind the scenes. A fellow colleague and friend, Miguel Madero, was also encountering similar issues. He also did some work in debugging the issues he was seeing, and because they were similar to mine, I had a little headstart in where to look. Note: I know that Miguel will also be publishing some debugging tips around Silverlight and he is a Silverlight gun so keep an eye on his blog for more details.

At any rate, our silverlight apps, after opening certain pages many times (> 8), the application would crash. No real information why, and the Visual Studio debugger wasn't helping a lot as the exception it was faulting on was just the final exception when things got too much.

In Visual Studio, turning on Catch all first chance exceptions wasn't helping either as there were too many other first chance exceptions that were confusing things (and yes, many framework related ones).

So, using WinDbg, we decided to dig deeper. The rest of this post shows how we debugged some silverlight issues using WinDbg using some general examples, and a few specific ones.

Getting Started.

You can create crash dumps to examine using a tool like adplus (part of the debugging tools) which I did, but I also prefer to attach to the running process itself where possible so I shall use that here.

1. Loaded up WinDbg

2. Loaded the SOS module for silverlight which is the managed debugging extension.

3. Set the symbol path to a local directory on my system as well as the microsoft symbol servers when the tool needs to download/resolve symbols.

You can do this via the command line

or via the user interface

4. Fire up your SIlverlight application in a standard Internet Explorer browser window. Do not invoke debugging within Visual Studio (ie. press the 'Play' button or hit 'F5' to start the app) as this will attach its own debugger. Then attach WinDbg to the 'iexplore.exe' process.

5. Now we are ready to begin actual debugging. Typing 'g' into the command line gets the debugee running, and WinDbg monitors the application for first chance exceptions.

One thing that is not so apparent within Silverlight, particularly when using the MVVM (Model-View-ViewModel) pattern which typically uses DataBinding extensively, is when data binding errors occur. The framework tends to hide these little gems away, and sometimes, it just appears like its not working. Note that these will also appear in the Visual Studio debug output window but I find it actually easier to have the WinDbg window running side by side with the browser to watch all exceptions as they occur.

By way of example, lets imagine the view model has this property, SelectedThing.

Now lets also imagine we have a ComboBox databound to this using the following XAML.

Finally, when we fill up/gather our data, we set the SelectedThing to an initial value:

SelectedThing = SomeData[0];

The astute among you can probably already see where the error is, but lets persist for the sake of example. We load the app and see that the ComboBox is not having its selected item bound to anything.

At this stage, no errors are prevalent, other than we have no SelectedItem in the ComboBox. Hpwever, looking at the WinDbg window, shows us that there were some exceptions occurring.

Specifically, we can see that an exception occurred with a BindingExpression, in which a converter failed to convert a value for 'SomeItem'. The entire message is shown below, and its pretty verbose about what failed.

(Again, this is the same output as the Visual Studio debug output window)

System.Windows.Data Error: 'MS.Internal.Data.DynamicValueConverter' converter failed to convert value 'TestSLCrap.SomeItem' (type 'TestSLCrap.SomeItem'); BindingExpression: Path='SelectedThing' DataItem='TestSLCrap.ViewModel' (HashCode=64479624); target element is 'System.Windows.Controls.ComboBox' (Name='DropList'); target property is 'SelectedIndex' (type 'System.Int32').. System.InvalidOperationException: Can't convert type TestSLCrap.SomeItem to type System.Int32.
   at MS.Internal.Data.DefaultValueConverter.Create(Type sourceType, Type targetType, Boolean targetToSource)
   at MS.Internal.Data.DynamicValueConverter.EnsureConverter(Type sourceType, Type targetType)
   at MS.Internal.Data.DynamicValueConverter.Convert(Object value, Type targetType, Object parameter, CultureInfo culture)
   at System.Windows.Data.BindingExpression.ConvertToTarget(Object value).

It tells us the value to convert, the binding path and the property. Pretty easy to fix. In the XAML we simply change this:

SelectedIndex="{Binding SelectedThing, Mode=TwoWay}"

to this:

SelectedItem="{Binding SelectedThing, Mode=TwoWay}"

Again, pretty simple and actually an easy error to do if you are going fast and accepting the Intellisense suggested property without really looking. To bring this back to my problematic silverlight app I mentioned, while debugging I found numerous instances of binding issues where properties were NULL when not expected and a few others. Eliminating these certainly made the app run better but it was still crashing.

So, I gotta dig deeper. I had a look at some of the exceptions on the heap.

0:005> !dumpheap -type Exception
Address MT Size
03a21024 038c72d0 72
03a2106c 038c7398 72
03a210b4 038c7458 72
03a210fc 038c750c 72
03a21144 038c750c 72
03a24edc 060a5668 32
03a25a4c 060d51e4 32
03a25a6c 060d52c0 32
03a2dac4 060dd240 32
03a3d254 0321f48c 12
03a3d26c 0321f510 12
total 11 objects
Statistics:
MT Count TotalSize Class Name
0321f510 1 12 System.Text.DecoderExceptionFallback
0321f48c 1 12 System.Text.EncoderExceptionFallback
060dd240 1 32 System.EventHandler`1[[System.Windows.ApplicationUnhandledExceptionEventArgs, System.Windows]]
060d52c0 1 32 MS.Internal.Error+GetExceptionTextDelegate
060d51e4 1 32 MS.Internal.Error+ClearExceptionDelegate
060a5668 1 32 System.UnhandledExceptionEventHandler
038c7458 1 72 System.ExecutionEngineException
038c7398 1 72 System.StackOverflowException
038c72d0 1 72 System.OutOfMemoryException
038c750c 2 144 System.Threading.ThreadAbortException
Total 11 objects

Hmmm, not too much interesting here as these exceptions. Lets dump the current stack objects to see what that yields:

0:005> !dso
OS Thread Id: 0xa54 (5)
ESP/REG  Object   Name
0179E85C 04084278 Views.DataDisplayView
0179E880 03a82684 System.Windows.CoreDependencyProperty
0179E884 04084278 Views.DataDisplayView
0179E89C 04084278 Views.DataDisplayView
0179E8BC 03a82684 System.Windows.CoreDependencyProperty
0179E8C0 04084278 Views.DataDisplayView
0179E8CC 0450f6d8 System.Windows.Controls.RadioButton
0179E8E0 03a82684 System.Windows.CoreDependencyProperty
0179E8E4 04084278 Views.DataDisplayView
0179E8F0 03a82684 System.Windows.CoreDependencyProperty
0179E8F4 04084278 Views.DataDisplayView
0179E904 04084278 Views.DataDisplayView
0179E910 04084278 Views.DataDisplayView
0179E914 04084278 Views.DataDisplayView
0179E918 04084278 Views.DataDisplayView
0179E91C 04088ca0 System.Windows.Controls.RadioButton
0179E938 04088ca0 System.Windows.Controls.RadioButton

. . . majority of listing omitted for brevity

0179E980 0450f6d8 System.Windows.Controls.RadioButton
0179E990 0455641c System.Windows.RoutedEventArgs
0179E994 0450f6d8 System.Windows.Controls.RadioButton
0179F7C4 03a6f9d0 Ninject.Core.Injection.DynamicMethodInjector
0179F7C8 0425fcd8 System.Object[]    (System.Object[])
179F9BC 03a92720 PresentationLayer.UI.UserControls.MenuItem
0179F9E4 03a92c00 System.Windows.Input.MouseButtonEventHandler
0179FA04 03a92720 PresentationLayer.UI.UserControls.MenuItem
0179FA08 0425f898 System.Windows.Input.MouseButtonEventArgs
0179FA44 0425f87c System.String    M@21
0179FB24 03a2526c MS.Internal.FrameworkCallbacks+FireEventDelegate
0179FBC8 03a2526c MS.Internal.FrameworkCallbacks+FireEventDelegate

Now this is much more interesting, as you can see multiple references to Views.DataDisplayView where only 1, should be present. I poked around the objects themselves and found similar patterns. Using the 'gcroot' command, I could see what objects were 'hanging around'.

0:005> !gcroot 04084278
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Scan Thread 5 OSTHread a54
ESP:179e85c:Root:  04084278(Views.DataDisplayView)->
ESP:179e884:Root:  04084278(Views.DataDisplayView)->
ESP:179e89c:Root:  04084278(Views.DataDisplayView)->
ESP:179e8c0:Root:  04084278(Views.DataDisplayView)->
ESP:179e8e4:Root:  04084278(Views.DataDisplayView)->
ESP:179e8f4:Root:  04084278(Views.DataDisplayView)->
ESP:179e904:Root:  04084278(Views.DataDisplayView)->
ESP:179e910:Root:  04084278(Views.DataDisplayView)->
ESP:179e914:Root:  04084278(Views.DataDisplayView)->
ESP:179e918:Root:  04084278(Views.DataDisplayView)->
ESP:179e91c:Root:  04088ca0(System.Windows.Controls.RadioButton)->
  0408900c(System.Windows.DataContextChangedEventHandler)->
  04088fd0(System.Windows.Data.BindingExpression)->
  04076c74(Views.DataDisplayViewViewModel)->
  04076eb8(System.Collections.ObjectModel.ObservableCollection`1[[MyApp.DataEntities.WeekCommence,  
MyApp.DataEntities]])->
  0409dbb0(System.Collections.Specialized.NotifyCollectionChangedEventHandler)->
  04085a34(System.Windows.Controls.ComboBox)->
  04085c84(System.Windows.Controls.SelectionChangedEventHandler)->
  04084278(Views.DataDisplayView)->
ESP:179e938:Root:  04088ca0(System.Windows.Controls.RadioButton)->
  0408900c(System.Windows.DataContextChangedEventHandler)->
  04088fd0(System.Windows.Data.BindingExpression)->
  04076c74(Views.DataDisplayViewViewModel)->
  04076eb8(System.Collections.ObjectModel.ObservableCollection`1[[MyApp.DataEntities.WeekCommence,  
MyApp.DataEntities]])->
  0409dbb0(System.Collections.Specialized.NotifyCollectionChangedEventHandler)->
  04085a34(System.Windows.Controls.ComboBox)->
  04085c84(System.Windows.Controls.SelectionChangedEventHandler)->
  04084278(Views.DataDisplayView)->
Scan Thread 13 OSTHread 16b4
Scan Thread 14 OSTHread 1274
Scan Thread 15 OSTHread 15a8
Scan Thread 16 OSTHread 1330

I could see many instances of references to properties of type ObservableCollection<>. Looking at the pinned/rooted objects for a view other instances of the 'Views.DataDisplayView' objects, revealed similar results, some showing many instances of references to properties that implemented ObservableCollection<>.

Obviously, something was subscribing to the collection changed events of the ObservableCollections, and simply not letting go. Whether this be the DataBinding infrastructure of Silverlight or a result of some of the dodgy code in the application, its currently hard to say as the code is quite convoluted.

I can say this however. Changing the properties listed in the WinDbg sessions from ObservableCollections to simple generic List<> types completely solved my crashing issue. Simply changing these types involved a bit of work to now manually raise the appropriate property changed events when these properties are changed, but at least my application doesn't crash.

As a result, I now keep WinDbg loaded during most of my silverlight development, with the symbol path and symbols loaded and ready to go. its easy to just attach to the iexplore.exe process, make sure its all good, and be on my way. I know that WinDbg can save all this workspace information, but for some reason, sometimes it doesn't so I generally keep it loaded and its small enough anyway.

Hope this proves informative for someone.

WCF, IErrorHandler and propagating faults

Glav — Thu, 16 Oct 2008 10:12:00 GMT

On a recent project, I have needed to set up a generic error handler just above the service layer which ensured that only known exceptions were propagated up to the client in a controlled manner. This post contains information about how to write an IErrorHandler, but more importantly, it does contain an important point at the end about its implementation that I wanted to share, as well as save for my own reference on this blog, in this wonderful inter-web thingy.

Update: The code for this article is here

Using FaultContracts is a good way of achieving this WCF, but I also wanted to be able to:

Catch generic CLR exceptions and propagate them to the client as a generic fault exceptions.
Catch some NHibernate specific exceptions (the concurrency ones in particular) and propagate these to the client as a general concurrency fault.

This was all the client needed to know, and also provided an opportunity to log any unhandled exceptions that may sneak there way up to the services. WCF provides great exception shielding by default, but we wanted a bit more control.

Enter the IErrorHandler interface that allows you to intercept any faults or exceptions before they are sent back to the client. To use this properly, you still need to define your FaultContract as required. My simple example contract is as follows:

So to start the implementation of my custom error handler, I define a class that will implement an IErrorHandler as well as IServiceBehavior.

I do this because not only do I want to create an IErrorHandler implementation, but I need it to participate as a behavior within the WCF pipeline, that is, I want to add it in as a ServiceBehavior when I create the host so that my custom error handler implementation can be plugged in.

I first provide implementations for the IErrorHandler.

HandleError returning true simply means I have handled this error and don't need other error handlers to deal with it.

ProvideFault does a few things. First it checks if its a genuine FaultException. If so, it does nothing but let it pass through. If it isn't, then a FaultException<ApplicationFault> is created, and a message fault is generated from that. Then a new message is created, using the message fault is its content, and that message is assigned to the return fault parameter.

If I wanted to catch another type of exception, I might check for it something like:

if (error is NHibernate.StaleObjectException) .....

and optionally create a different type of fault. The ApplicationFault used to construct the FaultException is just a standard DataContract.

Implementing the IServiceBehavior is pretty easy as you see below:

If you wanted to, in the Validate method, you could iterate through each operation and ensure its got your required FaultContract applied like this:

As a nice touch, I wanted this error handler to be able to added via configuration and play nicely with the WCF Service Configuration editor. To do this, you need a class that inherits from BehaviorExtensionElement and implements 2 simple methods like this:

So, finally we are at the point that caught me up for a while. In the implementation of the ProvideFault method of my error handler, you need to specify an action for the returned Fault as shown below:

Now its important that this Action is a known action identifier that your client is expecting. In other words, if you generate a proxy (via SvcUtil) for this service and peek into the reference.cs file for the 'Action' attribute on your services fault contract, make sure they are the same. Otherwise, when your error handler generates a fault with an unknown fault action, the client wont accept it as a known fault and it will look like just a generic FaultException, not the specific FaultContract<YourType> you were after.

When I was developing this error handler, I simply fudged this value while I got on with the task of implementing the handler. I forgot about it, then spend some time wondering why my faults were not being propagated and caught properly on the client. So make sure you supply a well known action for your fault in the contract attribute and in the error handler like this:

First on your FaultContract attribute:

Then in your ProvideFault implementation:

So finally, thats it. Don't forget about matching up those actions, otherwise you'll waste your time.

Hope this has proved helpful and informative.

Update: The code for this article is here

EOAST - Evolution of a software thingy - Part 3

Glav — Tue, 09 Sep 2008 00:54:42 GMT

This is the the third instalment of my experiences and ideas around the development of a piece of software to initially satisfy my disappointment around Outlook and also to allow me to play with some new technologies and making them all work together. Part 1 is here and Part 2 is here.

This latest instalment will focus a little more into the design detail. The "InformationAggregator" (as I have coined this project) is intended to be fairly generic as to the kind of information it collects but is still somewhat specific in that all types of information ever thought of is a bit too grandiose. For the most part, email, RSS, meetings, tasks, journal entries, notes and those types of things are the initial concern.

Central to this is the "GenericInformationItem" that is used to house individual elements of information. Below is a class diagram of the object:

I'll go through the non obvious items to explain how I see them working. Note that this is a work in progress, so things are bound to change as I go further down the path.

BodyBinaryData and BodyTextData: These properties are obviously the body component of the information item. BodyBinaryData might typically be images or pretty much anything you like in binary form. It does not lend itself to searching its contents though, and this is somewhat mitigated by searching on things like Title, Author & Summary fields, but this still may not be quite sufficient. BodyTextData is any textual portion of the information item body and is easily searched. Email body, RSS item content, note content are obvious candidates here.

InitialDateTime and EndDateTime: My thinking here was to mainly cater for meetings/appointments. InitialDateTime can also be used for the start or creation date/time (depending on what information item is being modelled). EndDateTime may not always be used but could also be adopted for expiry dates (again, dependent on the information item being modelled and contextually known about by the information provider and display modules).

SourceProvider: This is the name of the InformationProvider module that provided the InformationItem. Queries and mapping against InformationDisplay modules work against this property. Only registered InformationProviders will have their data accepted and stored in the repository.

LastNotification, NextNotification and NotificationRequired: These fields control whether this module and associated collected information items support notification and any display providers should take this into account and display a notification when an item requires notification. The way a notification is display (if at all) is at the discretion of the InformationDisplay module.

InformationSource: This property indicates the original source of the information. This might be a URL, file location, email account or anything.

Status: This property indicates if the InformationItem is in fact an error item. These items typically aren't stored in the repository as normal information but are used to communicate errors to various modules and the engine itself, and are reported in event logs, traces etc.

So that's a small insight into the detailed design of this software thingy aka InformationAggregator

The next instalments will begin the focus on the interface implementations and how I see data going back and forth to the engine. Hopefully its not too boring, and as I mentioned, t his is a work in progress. I welcome any critique, suggestions and feedback.

TechEd Oz 2008 Presentation Demos

Glav — Mon, 08 Sep 2008 06:12:59 GMT

Well TechEd Au 2008 is over and done with. I did a presentation called Productive WCF and as promised to all the delegates that attended my presentation (thank you very much for attending by the way), I have provided a link where you can download a ZIP file containing all the demo code.

You can download the code from here.

I hope everyone enjoyed TechEd and look forward to seeing everyone at the next big Tech event.

EOAST - Evolution of a software thingy - Part 2

Glav — Mon, 11 Aug 2008 05:14:59 GMT

In a part 1 of this series, I talked about what was the purpose of coming up with this "software thingy". Now I will tackle some of the design aspects that have evolved as I have been working on it.

Just to mention the requirements again though, it must be able to gather and store information from literally any source, and not tied to one collection mechanism. It must also be able to display any data in any format, typically something that the user/administrator configures to their liking. It must be easy to develop the collection and display parts. The main engine will take care of scheduling collections and storage of the data. Data can be pushed to it, not just collected. Must be reliable and "non blocking' in the event of a bad collection or display module.

It needs to allow me to exercise some of the new fandangled technologies but not at the expense of functionality.

Whew. Thats it in a nutshell. So what have I come up with so far (remember, being a pet project, its really a moving target). The diagram below shows a conceptual diagram of what needs to occur and the logical boundaries of components.

Obviously, a common element to house each item of information would be a good way to go initially. Something that caters for all types of information, with indicators pertaining to its source. As long as the data can be stored, and ideally searched, the display of that item is almost irrelevant for now.

Here is a sketching of the information items I initially wanted to capture/cater for.

So with the above things in mind, it seemed appropriate that I dub this piece of software the "Information Aggregator". Yes, like an RSS aggregator but with general information item in mind, which may or not be RSS items.

Continuing with the high level design though, extrapolating the above 2 diagrams further, and taking into account how I want the entire process to operate, below is an initial diagram of some components/services and how I expect interfaces and information flow to occur.

You can see that the "Engine" is the core component. The idea is that it maintains a list of all collection agents and display agents. The engine will co-ordinate scheduling of collections from the various agents, and also allow registered agents to push information to the engine.

Separately, upon receiving information or based on request from display agents, the engine will push information to any display agents, that have been associated with a particular collection type.

You can also see the interfaces that I have begun to design here. I will delve more into this in the next post, however the main interfaces being IPluginModule, IInformationPush and IInformationPull. IPluginModule being the interface that all compliant modules must implement. IInformationPush and IInformationPull are optional (although at least 1 makes sense) depending on the type of the module.

Note: I am aware you could probably use one of the composite application blocks to achieve an outcome similar to what I describe here, but I didn't want to rely on anything like that. Mainly because this is about learning and playing with stuff, as much as getting to an end solution.

So the eventual concept being built, is that information gets collected from some collection providers, lets say a provider called GlavsEmail and GlavsNoteEntry.

I have 3 registered display providers:

A balloon notification
A full detail display
A summary display.

I map the GlavsEmail information provider to the balloon notification and the summary display. I map the GlavsNoteEntry to the full detail display.

I set up the schedule for the GlavsEmail collection to occur every 1 hour. I set the schedule for the GlavsNoteEntry o be manually initiated (ie. by the information provider itself).

The engine should now perform the necessary collections when required, and when elements of GlavsEmail are gathered, display those using the 2 mapped display providers (balloon and summary), and the GlavsNoteEntry gets displayed using the full detail display.

At any time I can change this mapping. Also, all of the information from those sources is stored in SQL for easy searching and extraction if required. All I would typically have to do is register the collection and display providers, map them accordingly, and set up the access credentials for the collection provider (email in this case).

Thats it for this post. Next post I will delve more into the design of the interfaces, and technical details of the software (might be a few posts actually). Mind you, its still a moving target and not complete (like most of my personal pet projects) so don't expect enterprise level detail and a complete publicly consumable API.

The post after that will probably show you the product in action :-)

EOAST - Evolution of a software thingy - Part 1

Glav — Mon, 04 Aug 2008 05:48:10 GMT

Its been a little while since I have blogged as I haven't had very many value items to add to the blogosphere lately. However, I have been steadily (albeit slowly) working away on a little software project that may or may not see the light of day, but it has been a fun learning experience, and may eventually come to something. If nothing else, its a good testbed for a number of latest technologies.

I decided to chronicle some aspects of this software "thingy", hence the blog title "Evolution of a software thingy" - EOAST (you gotta have an acronym otherwise you don't exist :-) )

So this first post is the requirements if you will, the business drivers behind what and why I decided to do what I did.

Context

If you have read any of my previous rants, you'll note that I am not the biggest fan of the latest incarnation of outlook. I use it daily, and rely on it, but its slow, often locks up (particularly on slower links) and generally sucks the life from my system.

Additionally, like most people, I use a number of various tools to collect information for either storage, perusal, reference and a bunch of other things.

The collection of various information pieces shows a lot of similarity with each other, the main difference being collection frequency or scheduling, management of the information, and its presentation. For all the tools I use, none of them are exactly how I want them. Each one has different parts I like and dislike and merging the information they gather into a manageable unit is tedious, if not almost impossible in some cases. Think blog aggregators, email, twitter etc. Essentially, they all collect some information at different frequencies or schedules and display it for manipulation or searching, sometimes storing it for later retrieval or whatever.

That's the pain point part. I also wanted to have a purpose for some of my technological investigations. We all know that you can play with a new technology like Entity Framework, or Linq to SQL and make it do what the examples do. However, when hard pressed to do something that meets a requirement, not a technology demonstration, and suddenly the game changes, and efficiencies and/or deficiencies are far more apparent. So essentially I wanted to play with some new tech as well to get a real feel for its capabilities.

Requirements

So with that in mind, my requirements for my "Software Thingy" are as follows:

Must be able to collect information at various scheduling intervals, whether monthly, daily, hourly, every minute, or manually initiated. Additionally, it must also be able to push information, rather than simply request via a schedule. Think Microsoft exchange (push) and blog readers (pull).
it must store this information that it collects into a reliable store, SQL Server being the obvious candidate. This will cater for large volumes and easy searching. (Think problems with large Outlook PST files).
It must be reliable to handle badly written modules that it interfaces with. If something is taking too long, or just plain dead, the main application/engine will not die as well and render itself unusable (ummmm....Outlook again :-) ).
It must not be tied to one method of collecting information. It must be flexible enough to gather information from anywhere. Blogs, twitter, POP email, exchange email, manual entry into some data capturing app, text files, anything.
it must not be tied to one display method. I want the ability to determine how particular information is displayed. I'd like to be able to map the gathering of information from one source, to a certain display method, or even multiple display methods, all administratively.
I don't want to do it all. I don't have that much time or smarts. I would like to make something very easy to develop display modules or information gathering modules so that I can get perhaps a WPF guru to make me a slick WPF display module, without having to care where the info comes from and how its collected. Similarly for the information gathering. I'll write the items that pertain to me, and allow others to easily bolt on a different form of information gathering and not necessarily care about how its stored nor displayed.
I want to utilise technologies such as WCF (my favourite), Linq to SQL, WPF and any other thing that makes sense.

Next up will be the initial design that I have come up with. Stay tuned.

MSFeedsSync - feeds Synchronization has stopped working issue

Glav — Sun, 03 Aug 2008 10:59:30 GMT

Recently (and quite randomly I might add), I had an error dialog popup continuously, every 5-10 minutes which read:

"MSFeeds Synchronization has stopped working (2054)"

It was driving me crazy and was unsure how to resolve. There is actually not that much on the net about but this post got me going in the right direction.

Basically, I first disabled the service by typing at the command prompt:

msfeedssync disable

Then I deleted all my files (which included many feeds I was no longer using) from the following dir:

\Users\%username%\AppData\Local\Microsoft\Feeds

then re-enabled the feeds service by typing:

msfeedssync enable

And all was good again.