Enforcing a Build and Test Policy

I have just realised that VS.NET has everything in place to support a build and test policy without the use of any external add-ins! You can define and enforce your unit tests in code on a per project basis. This technique uses the little known ‘ComRegisterFunction’ attribute and ‘Register for COM interop’. If this method returns an exception then the build fails. A new app domain is created per build with a base directory of ‘bin\Debug’.

Here is an example that uses the NUnit test runner (although any test runner would do). As well as stopping the build when any tests fail, test results will be written to an XML file. Tests will only be enforced on a ‘Debug’ build. The whole thing could have been done in a few lines, but I decided to add exception handling and a few comments. ;o)

using System;
using System.IO;
using System.Runtime.InteropServices;

// By default make types invisible to COM
[assembly: ComVisible(false)]

#region Run tests after every DEBUG build
#if DEBUG
namespace Tests
{
 using NUnit.Core;
 using NUnit.Framework;
 using System.Diagnostics;

 // The test runner must be COM visible
 [ComVisible(true), ProgId("SampleTests")]
 public class TestBeforeRegistering
 {
  // NOTE: "Register for COM interop" must be enabled
  [ComRegisterFunction]
  public static void abortBuildOnTestFailure(Type t)
  {
   try
   {
    string path = new Uri(t.Assembly.CodeBase).LocalPath;
    TestSuiteBuilder builder = new TestSuiteBuilder();
    TestSuite suite = builder.Build(path);
    TestResult result = suite.Run(new NullListener());
    writeXml(path + "_tests.xml", result);
    if(result.IsFailure) { throw new Exception("Test Failure!"); }
   }
   catch(Exception e)
   {
    Debug.WriteLine(e);
    throw;
   }
  }

  // Write test results to an XML file
  private static void writeXml(string path, TestResult result)
  {
   XmlResultVisitor visitor = new XmlResultVisitor(path, result);
   visitor.visit((TestSuiteResult)result);
   visitor.Write();
  }
 }

 // An example failing test
 [TestFixture]
 public class SampleTests
 {
  [Test]
  public void FailTest()
  {
   Assertion.Fail();
  }
 }
}
#endif
#endregion

I have used the AylarSolutions.Highlight online demo for the code highlighting. If anything is broken blame Thomas. ;)

To start testing after every build all you need to do is enable 'Register for COM interop'.  This has the 'interesting' side effect of registering the test runner class as a COM object with a ProgID of 'SampleTests'.  It would be very strait forward for unit testing frameworks to support the running of named test suites!  Unfortunately there is no tidy error message when the build/test fails. All you get is " COM Interop registration failed. Exception has been thrown by the target of an invocation". Feel free to post any refinements of the idea/code you come up with.