IBlog<Johan>

How to do Custom Event Logging and Trace Writing in Azure

We’re about to begin our first “real” Azure project and I needed to wrap my head around event logging and trace writing in the cloud. Any proper application running in any cloud needs this sooner or later.

I quickly noticed that the information around Azure diagnostics was all over the place, and some things seems to have been changed in newer versions of the Azure SDK. So, something I thought was going to take me just a few minutes to figure out took about a day, but I think I got it sorted in the end and here’s what I did to get simple event logging and trace writing to work for our web role.

First thing I had to understand was how data for all kinds of diagnostics is handled and stored in Azure. You need to define which kinds of diagnostics data you want to capture, and you need to define where Azure should transfer the data captured so that you can download them or look at then with various tools. You also need to define how often these values should be transferred. I’ll get back to that a bit later.

Create a Custom Event Source

First thing I wanted to have was a custom event source to write to for warnings and errors. To do this I created a simple cmd-file which I placed inside the web role project and uploaded with the packed deployment. Now when the web role is deployed, you define startup task which runs in elevated mode (to be able to create the event source) and points to this cmd-file. So, create a file inside your project’s root called CreateEventSource.cmd and put this inside it:

EventCreate /L Application /T Information /ID 900 /SO "MySource" /D "my custom source" 

NOTE! Make sure the cmd-file property for “copy to output directory” is set to “copy always” or “copy if newer”.

Then open up your service definition file, ServiceDefinition.csdef, and add a startup task to it:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="MyWeb" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
  <WebRole name="MyWebRole" vmsize="ExtraSmall">
    <Startup>
      <Task commandLine="CreateEventSource.cmd" executionContext="elevated" taskType="simple" />
    </Startup>
    <Sites>
… and so on…
</ServiceDefinition>

So now when your application is deployed the cmd-file should be executed with elevated permission and create the event log source you specified. EventCreate is a tool which is available on the Azure server so it’s nothing you need to worry about. Same thing should work well on your local dev machine too.

Storage Location for Diagnostics Data

You need a place for the diagnostics data to be transferred to. This is number of Azure Storage tables so you need to create an Azure Storage Account where Azure can create these specific tables and transfer the captured data. So you basically go to the Azure Management Portal and create a new Storage Account and copy the access key. Then paste the storage account name and key into the Service Configuration:

Now when Azure starts to transfer data from event log and trace writes, they will be written to these two tables; WADLogsTable and WADWindowsEventLogsTable:

If you capture performance counters and such, these will be written to other tables.

Diagnostics Configuration

Next thing to do is to configure which diagnostics to capture for your application. This is done in the OnStart() method of your WebRole.cs file. What I wanted to test initially was event logging to my custom source and also some trace writings for information, warning and errors. I also specified how frequently I wanted these diagnostics to be written to the storage tables:

public override bool OnStart()
{
    var diag = DiagnosticMonitor.GetDefaultInitialConfiguration();
            
    diag.Logs.ScheduledTransferLogLevelFilter = LogLevel.Information;
    diag.Logs.ScheduledTransferPeriod = TimeSpan.FromSeconds(30);

    diag.WindowsEventLog.DataSources.Add("Application!*[System[Provider[@Name='MySource']]]"); 
    diag.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromSeconds(30);

    DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diag); 

    return base.OnStart();
}

Note a few things about this code – The transfer schedule is set to 30 seconds and you may want to set this one a bit higher and also look at the buffer quotas. The sample code is also only capturing and transferring event logs from the “Application” logs, and I’ve added an xpath expression to only transfer logs from the “MySource” event source we specified in the cmd-file earlier. You may want to grab all application events as well as all system logs too, in that case:

diag.WindowsEventLog.DataSources.Add("System!*");
diag.WindowsEventLog.DataSources.Add("Application!*");

Writing to Event and Trace Logs

The simplest thing is then to add some code to write to the event log and trace logs in your code, for example:

Trace.TraceError("Invalid login");
EventLog.WriteEntry("MySource","Invalid login",EventLogEntryType.Error);

Looking at the Data

To actually see what’s been written to these tables, you need to either download the data from the storage tables via APIs, use the server explorer or some tool, like the one from Cerebrata – “Azure Diagnostics Manager” – which seems to do the job pretty well. As far as I know, there is no proper tool available from Microsoft. Be that as it may.

Please, feel free to write comments below and add tips and tricks about handling diagnostics in your Azure application. I’d like to know more about the Transfer Period and Buffer Quotas and how they may affect the application and maybe also the billing… I’m sure there are loads to learn about this .

How to Force Comments When Doing Check-In in TFS

I was looking for a policy like this, and found this blog article by Claus Konrad:

http://blog.clauskonrad.net/2010/08/tfs-2010-how-to-force-comments-when.html

Now, unfortunately everyone in the project has to have the TFS Power Tools installed to make this work.

The power tools can be downloaded from the Visual Studio Gallery, this is a direct link to it:

http://visualstudiogallery.msdn.microsoft.com/c255a1e4-04ba-4f68-8f4e-cd473d6b971f?SRC=VSIDE

Whish this was in the box from start.

MSTest and Calling Exe with Sample Data Files

For various reasons I’m using MSTest for my unit tests and I have this console app which generates a PDF file from an XML file that needed some tests.

So, when moving the unit tests over to a TFS build server, having hard coded paths to test data files is not a good idea. The recommended way of doing it is to use the DeploymentItem attribute.

Also, when calling an exe-file which does Environment.Exit() and catching the exit code from the unit test is pretty straight forward by using Process.Start() and checking the ExitCode property.

A small sample:

[TestMethod]
[DeploymentItem(@"TestData\arbetsorder.xml", "TestData")]
[DeploymentItem(@"TestData\arbetsorderReport.rdlc", "TestData")]
public void GeneratorShouldReturnWithExitCode0()
{
    Process proc;
    try
    {
        proc = Process.Start("PdfGenerator.exe", "\"" + Directory.GetCurrentDirectory() + "/TestData/arbetsorderReport.rdlc\" \"" +
            Directory.GetCurrentDirectory() + "/arbetsorder.pdf\" \"" + Directory.GetCurrentDirectory() + "/TestData/arbetsorder.xml\"");
    }
    catch (System.ComponentModel.Win32Exception ex)
    {
        proc = null;
        Assert.Fail(ex.Message);
    }

    Assert.IsNotNull(proc);
    proc.WaitForExit(10000);
    Assert.IsTrue(proc.HasExited);
    Assert.AreEqual(0, proc.ExitCode, "Expected 0, got " + proc.ExitCode);
}

But, there’s a small caveat to get the DeploymentItem argument working properly! First you have to open up the test settings of your .testsettings file, select the Deployment settings and make sure the “Enable deployment” checkbox is checked:

That’s all you need to do. And note that you may actually have to exit and restart Visual Studio to make it work. I never got the files deployed until I did that restart.

Command Handling the Nancy Way

MiniBuss is a micro service bus framework over msmq which consists of less than 400 lines of code, sitting inside one single source file. The project is hosted over at http://minibuss.codeplex.com and the source code is maintained at https://github.com/johandanforth/MiniBuss

I’ve been a fan of the Sinatra inspired web framework called Nancy, especially the neat way of setting up handlers for routes. The simplest sample on their site is this:

public class Module : NancyModule
{
    public Module()
    {
        Get["/greet/{name}"] = x => {
            return string.Concat("Hello ", x.name);
        };
    }
}

So, I shamelessly dug into the Nancy code and borrowed some 20-30 lines of code and came up with something like this for registering handlers for certain incoming commands on the minibus, what do you think?

public class CommandHandler : MessageHandler
{
    public CommandHandler()
    {
        WhenReceiving[typeof(SampleMessage1)] = x =>
        {
            Console.WriteLine("sample 1 message: " + x.Text);
        };

        WhenReceiving[typeof(SampleMessage2)] = x =>
        {
            Console.WriteLine("sample 2 message: " + x.Text);
        };
    }
}

It’s a bit more code but it helps/enforces you to move the handlers off to a certain module. Thoughts?

More MiniBuss Updates

MiniBuss is a micro service bus framework over msmq which consists of less than 400 lines of code, sitting inside one single source file. The project is hosted over at http://minibuss.codeplex.com

Thanks to @CodingInsomnia for testing out the MiniBuss stuff a bit more than I did For the samples, and for my initial testing code, I used a shared assembly with messages (events and commands), which shouldn’t be necessary. So I made a few simple changes and now you can choose to either share messages in an assembly between your sender/receiver and publisher/subscribers OR you can declare local message classes as long as those classes use the same class name and properties it should work.

The NuGet package has been updated, and the new code is in package version 1.0.2.0.

MiniBuss Updates

I made a small update to MiniBuss, the micro service bus framework. The messages you send, Commands and Events, are no longer dependent on IMessage. The messages sent on the bus can now be any .NET class which can be safely serialized.

The MiniBuss package on NuGet has been updated (version 1.1.0.1).

I’m looking for testers, reviewers and co-authors. Areas I want to look more at are multithreading/concurrency and the reply-feature (especially being able to reply to current message in multithreaded scenarios.

MiniBuss on Codeplex

The micro service bus framework for msmq called MiniBuss is now open source on Codeplex at http://minibuss.codeplex.com

There is also now a NuGet package available for easy install into projects:

If you’re interested in co-op on this project, please contact me via Codeplex! I’m looking for devs and tester who knows their c#, msmq and concurrency.

A One File .NET Micro Service Bus for MSMQ?

This last year our company has invested quite some time in looking at CQRS, which led to looking at great looking service-buses like nServiceBus, Rhino Service Bus and Mass Transit, which led me to do some bus-coding on my own, mostly for fun and for learning MSMQ.

Inspired by the service buses mentioned above, the result became a bare-bones-one-file “micro service bus” that I call MiniBuss which sits on top of MSMQ. The file itself is only some 400 lines of code and supports send, receive, reply, publish and subscribe/unsubscribe.

Setting up a sender may look something like this:

Create the bus, register a message and tell it where messages of this type should go, start the bus and send the message.

Setting up a receiver may look something like this:

Create the bus and tell it which endpoint to listen to (which creates a local MSMQ queue if necessary) and tell it which message type to listen for and which delegate to kick off when such a message is received.

Similarly, when doing a receive/reply, you would have to create the bus on the sender side with a local endpoint and register a message-handler for replies, like this:

The receiver would do a bus.reply() like this:

The MiniBus also supports publish to multiple subscribers. A simple publisher would create a bus with a local endpoint (to receive subscribe/unsubscribe commands), tell it to handle subscriptions for a certain event, then start publishing something every 5 seconds (as an example):

Any clients interesting in subscribing to events from the publisher would create a bus with a local endpoint, start the bus and then send a subscribe command to the publisher, telling it you’re interested in subscribing to a certain type of event and which delegate to handle it:

Now, the question is, what do you think? I know there are issues with the code, like how to make message-handlers multi-threaded/concurrent without messing things up (it’s single threaded right now), and how to best handle exceptions, rollbacks and re-tries . Right now handling exceptions in send and receive works pretty well within a TransactionScope() together with ADO.NET, and if there’s an exception, the message is moved the an error-queue. No re-try or anything, only rollback and move to xxx_error. Also, the publisher doesn’t persist subscriptions, so if it is restarted subscribers wouldn’t know. You know, things like that.

I’m a user of the micro-orm called Dapper and like it a lot, so I’m thinking that maybe I should release this micro-bus as open source and see where people may take it, if anywhere? Maybe just down the drain because they figure out this service bus is dangerous to use and risk loosing messages or something (which would be extremely good to know :)

Or maybe this code is useless because you already got nServiceBus and Rhino out there and coders don’t need another service bus?

What do you say?

Pomodoro Timer for the Windows 7 TaskBar

EDIT:  The code is now open source on codeplex, on http://pomodorotaskbar.codeplex.com There's a new version available for download there too.

If you’re into the Pomodoro technique and looking for a pretty simple out-of-the-way timer which sits in the Windows 7 taskbar and is controlled from the Win7 JumpList, this one might be worth trying out.

NOTE: Icon overlays will NOT work if you're using small TaskBar icons!

That’s the timer looking like a tomato, with the remaining session minutes written out dynamically as icon overlay.

The timer has a couple of JumpList Tasks which you can use to control it. The program works well when “pinned” to the taskbar if you prefer it.

You can also control the timer from the main window with one large action-button (stop, start, stop ringing and restart session).

When the timer is stopped, you can just click on the remaining minutes-textbox and change the default session length to whatever you like. The value is stored.

When the session is ended, the window flashes, the tomato shakes and a ringing sound is played a few times to get your attention.

Note, this is a very simple program which I’m planning to release as open source when I’ve received some feedback. I prefer to keep it simple, but may add session logging. The name as well as the looks may change. .NET 4.0 is required.

Create Tag-Cloud from RSS Feed in ASP.NET MVC

Say you want to generate your own tag-cloud from a list of categories or tags you pull from an RSS-feed or similar. This is one way to do it. I’m using ASP.NET MVC for this sample which creates a simple tag-cloud in a Razor-view with HTML looking something like this:

The controller-code to start out with looks like this, where you read in the RSS-feed into a SyndicationFeed class, then pull out and flatten all the categories you have in that feed. Then group all those categories and count them to build up this simple view-model which is a Dictionary of category and percentage:

So now we got this view-model which is a Dictionary<string,int> and contains category/percentage data like this:

So all we need to do is create some kind of cloud in HTML or similar in the view. One way of doing it is this, which creates the output shown at the top:

Obviously, to be able to click on a tag and so on you need to create a richer view-model, I just wanted to show how I grab and count the tags from the feed