Moq.me 3.0 RTM

Congratulations Kzu on the release of Moq 3.0 RTM!

You can get it here http://moq.me/ (I love the domain name).

There’s a fascinating back and forth between Ayende and Daniel (kzu) where they discuss different ways to use mock frameworks. Ayende (who developed Rhino Mocks) is more in favor of interaction testing than Daniel who is a state/classic TDDer.

Recently I’ve been trying to separate my tests into unit tests, interaction tests and integration tests. For interaction testing I’ve been using Typemock Isolator, which allows me to break a few runes in order to exercise some hard to test code. Moq is looking very interesting as a mock framework to use with my traditional unit tests.

TestDriven.Net 2.19: Release Notes

Which Test Runner?

The latest version of TestDriven.Net will display the name of the test runner used in the test results summary. If the target project doesn’t reference a known test framework assembly, TestDriven.Net will use the ‘Ad hoc’ test runner to quickly execute the target method.

TestRunnerName

You can see the above method has been executed using the ‘Ad hoc’ test runner. If this happens when targeting real unit test, the chances are a test runner to handle the target test type or version hasn’t been registered.

NUnit 2.5 Beta 2 Support

This version includes support for NUnit 2.5 Beta 2. The previous version had a bug that caused tests that threw an exception to show as skipped rather than failed.

Test With > .NET 1.1 on a 64-bit OS

If the .NET 1.1 SDK is installed and you’re using Visual Studio 2005, you will have the option to ‘Test With > .NET 1.1’. Under the covers this functionality builds your project using MSBee before executing your tests using .NET 1.1. This will now work when executed on a 64-bit OS.

I’d be interested to hear if there’s any demand for this feature on Visual Studio 2008.

Visual Studio 2003 Support

This version once again works on Visual Studio 2003. Only one person noticed this was broken on the previous beta. Are there many projects still being developed using Visual Studio 2003? I’m using MSBee for all my legacy .NET 1.1 support.

Running xUnit tests on a 64-bit OS

Some people have been experiencing problems registering the xUnit test runner on a 64-bit OS. It turns out the problem was due to xUnit sometimes registering itself under HKLM and sometimes under HKCU depending on what was previously installed. If you’re experiencing this problem, try re-registering xUnit (using xunit.installer.exe) after installing this version.

Clear Error List Before Build

Failed tests will now be removed from the error list when a new build starts. This prevents failed tests from becoming mixed up with build errors. I think this is the correct default behavior.

Download

You can find the release notes here and download here.

Improved support for MbUnit, xUnit and Gallio

The main focus of the TestDriven.Net 2.18 release has been to improve support for test runner plug-ins in general (not just NUnit). If you’re using xUnit, MbUnit or Gallio – I recommend you upgrade to this version.

Automatic support for 64-bit machines

The registry layout on 64-bit machines is plain weird and full of pitfalls for the unwary. The registry layout is different depending on whether you’re installing under HKLM or HKCU. Under ‘HKLM’ the ‘SOFTWARE’ key is split and test runner plug-ins needed to be registered twice in order to work in both 32 and 64-bit processes. There is no such split under ‘HKCU’ and plug-ins installed there only needed to be registered once.

This created the unfortunate situation where plug-ins installed for ‘all users’ wouldn’t work when running in a 64-bit process, but plug-ins installed ‘just for me’ would work fine. Rather than expect plug-in developers to deal with this weirdness, I’ve made some changes to automatically support plug-ins that aren’t 64-bit aware.

If you have an assembly that needs to work on 32 and 64-bit machines, you may find the following snippet useful:

    public static RegistryKey OpenSoftwareKey(bool hklm, string name)
    {
        string fullName = @"SOFTWARE";
        if (hklm)
        {
            if (Marshal.SizeOf(typeof(IntPtr)) == 8)
            {
                fullName += @"\Wow6432Node";
            }
            return Registry.LocalMachine.OpenSubKey(fullName + @"\" + name);
        }
        return Registry.CurrentUser.OpenSubKey(fullName + @"\" + name);
    }

The following ad-hoc test will display TestDriven.Net’s install directory (assuming TestDriven.Net is installed ‘for all users’):

    void test()
    {
        using(var key = OpenSoftwareKey(true, @"MutantDesign\TestDriven.NET"))
        {
            Console.WriteLine(key.GetValue("InstallDir"));
        }
    }

Better support for ah-hoc tests

In previous versions of TestDriven.Net, a test runner plug-in was required to explicitly signal when none of its tests were found for execution. This would give other test runners (such as the ad-hoc test runner) a chance to handle the test. Unfortunately most test runners have been signaling a successful test run when tests were found but none were targeted.

I’ve changed it so the ad-hoc test runner will automatically get a chance to execute if no tests were executed and the test runner plug-in indicated a successful test run. The upshot of this is that you can now have ad-hoc side-by-side with MbUnit or xUnit tests.

If you’re using xUnit, try doing ‘Run Test(s)’ on each of the following methods:

    [Fact]
    public void TestMe()
    {
        Console.WriteLine("Console output isn't displayed when using xUnit");
        Assert.True(false, "Comment out [Fact] and run as ad-hoc test! ;)");
    }
    void hello()
    {
        Trace.WriteLine("Hello, World!");
    }
    void dump()
    {
        Trace.WriteLine(AppDomain.CurrentDomain, "_verbose");
    }

(ad-hoc tests should work side-by-side with all other test framework methods as well)

Improved performance when executing with Gallio / MbUnit v3

Gallio is a test runner that supports many different test types (MbUnit, xUnit, NUnit, MSTest and more). It has its own plug-in architecture and it doesn’t use the default TestDriven.Net app domain test isolation. This makes Gallio very flexible, but it also meant it wasn’t appropriate to setup and tear down the Gallio engine for each test run.

I’ve made some changes to allow Gallio to stay resident in the test process. This has significantly improved performance (especially for short test runs). If you’re using Gallio/MbUnit v3, try upgrading to Gallio v3.0.5 build 546 and see how much of a different it makes.

Feedback…

There have been lots of other changes which you can find in the release notes. If you notice any new issues, please don’t hesitate to let me know!

TestDriven.Net Options Pane

options

In the latest version of TestDriven.Net you will find a new options pane. The options are as follows:

Hide trace/debug output when running all tests in project/solution

In the past this setting has always been hardwired to true. It means you can add trace information to a test without cluttering up the ‘Test’ output pane when all tests are executed. An alternative way to ensure a message will always appear in the output is to use ‘Console.WriteLine’ rather than ‘Trace.WriteLine’. This also means the verbose Gallio/MbUnit test run output will show up when executing all tests in a project.

Cache test process between test runs

By default the external test process will be cached when the ‘Run Test(s)’ command is used. This process appears in the tool tray as a rocket icon which can be used to kill the process. This is fine unless one of your tests starts leaking leaking native resources (such as leaving open a file handle). The best solution is to fix the resource leak, but you now have to option to work around the issue by killing the test process at the end of each test run. This can be useful if the resource leak is in a 3rd party DLL which can’t be easily be changed.

Categories

This option supports the most common use of test categories; you can choose to include or exclude a selection of categories. This is useful if you want to exclude long running tests or if your machine isn’t configured to execute integration tests. This feature is currently only supported by the NUnit runner that comes with TestDriven.Net, but I believe it will be supported by a a future version of Gallio.

You can specify your NUnit test categories like this:

    [Test, Category("LongRunning")]
    public void LongRunning()
    {
        Thread.Sleep(10000);
    }

There are lots of features in TestDriven.Net that could be exposed as options. I’ll try to resist this temptation as much as possible, but I’m sure a few more will creep in. ;-)

TestDriven.NET 2.18 + NUnit 2.5 Beta

I’ve just uploaded a new version of TestDriven.Net with support for NUnit 2.5 Beta.

There’s a menagerie of weird and wonderful new attributes to choose from in this point release of NUnit. Ben Hall has written a good summary of the Alpha version and I’m sure Charlie Poole will be blogging about the Beta in the coming days.

Here’s a quick summary of a few new attributes:

Generic test fixtures can be used when you need to run a batch of tests against few different implementations of a type. In the past something similar could be achieved by having an abstract base fixture and extending it for each implementation you needed to test. By using a generic fixture you keep all your test code in a single class and I think it’s more expressive.

    [TestFixture(typeof(ArrayList))]
    [TestFixture(typeof(List<int>))]
    public class IListTests<TList> where TList : IList, new()
    {
        [Test]
        public void Count()
        {
            IList list = new TList { 1, 2, 3 };
            Assert.AreEqual(3, list.Count);
        }
    }

The ‘TestCase’ attribute is similar to MbUnit’s ‘RowTest’. With this attribute you transform a single test method into multiple test cases. You can also define an expected return result, but I’d advise against using this if you want a stack trace when your test fails. It’s better to explicitly define the assert inside the test method.

    public class TestCases
    {
        [TestCase(4, 2, 2)]
        [TestCase(2, 1, 1)]
        [TestCase(5, 2, 3)]
        public void Add(int answer, int a, int b)
        {
            Assert.AreEqual(answer, a + b);
        }
    }

In previous versions of NUnit you could specify which threading model your tests required by adding some XML to your test project’s App.config file. You can now specify this directly on the test that needs it using the RequiresMTA/STA attributes.

    [Test, RequiresMTA]
    public void MTA()
    {
        Assert.AreEqual(ApartmentState.MTA,
            Thread.CurrentThread.ApartmentState);
    }
    [Test, RequiresSTA]
    public void STA()
    {
        Assert.AreEqual(ApartmentState.STA,
            Thread.CurrentThread.ApartmentState);
    }

Lastly the ‘TestFixture’ attribute is no longer required and test methods are allowed to be static. This means NUnit can now be used in a natural way for testing F# code. Note, you will need to have "Other Flags" set to "--optimize+ notailcalls" in your project’s build properties if you want to see a stack trace on any failed asserts.

    #light
    open NUnit.Framework
    [<Test>]
    let fsharp() = 
        Assert.AreEqual(2 + 2, 4)

Update: I’ve tried to highlight a few features in NUnit 2.5 which can be used to make your unit tests clearer. Xerxes Battiwalla has written a post about Assert.Throws<T>() which also falls into this category.

    [Test]
    public void CreateDomain_Null()
    {
        Assert.Throws<ArgumentNullException>(() => AppDomain.CreateDomain(null));
    }

For more information see the TestDriven.Net 2.18 release notes and the NUnit 2.5 documentation.

Microsoft F# and TestDriven.Net 2.16

Now that F# is being officially productized, I thought it was time to make it a first class citizen inside TestDriven.Net.

When learning a new language I like to experiment by writing ad-hoc tests and viewing my code inside .NET Reflector. My initial focus has been getting 'Go To Reflector' and the targeting of parameterless methods and properties working.

You can get the latest release of F#, including the compiler, tools, and Visual Studio 2008 integration from the F# Developer Center. You will also need TestDriven.Net 2.16 or above.

The F# team has put together a collection of F# sample code. The samples come with their own 'F# Micro Samples Explorer' application, but they are also ideal for running as ad-hoc tests. In particular have a look at the 'beginners.fs' and 'intermediate.fs' modules in the 'Samples101' project.

Samples101

To view the code using .NET Reflector, simply right click inside a method, property, module or project and 'Go To Reflector'.

ReflectorFS

At the moment only parameterless methods and properties are supported. This is due to the way F# makes extensive use of type inferencing for parameters. Luckily it is parameterless methods and properties that it makes most sense to target for evaluation or running as unit tests. I'll talk more about unit testing F# code in a future post.

property

Hello, F#!

hellofs 

This is a quick post to break my writers block.

download

Silverlight NUnit Projects

I've been doing some experiments to find out if it's possible to execute NUnit on a Silverlight project. I knew that Visual Studio 2008 and Expression Blend both host Silverlight for use in the designer window. It turns out that rather than host a separate instance of the CoreCLR, the designer simply loads the Silverlight assemblies into the host runtime. This simplifies the business of creating a compatible unit testing framework considerably!

With a little bit twiddling, I've managed to tweak the 'nunit.framework' assembly so that it's compatible with Silverlight projects. In order for your tests to execute successfully, you also need to ensure that all Silverlight assemblies (except 'mscorlib') are set to 'Copy Local: True'. After this is done you can run, debug and even do code coverage on your Silverlight unit tests!

Because creating one of these projects is a fiddle to set up, I've created Silverlight NUnit Project template that does the work for you. To install the template simply open the 'SilverlightNUnitProject.vsi' file and ignore the unsigned content warning. Once installed, you will find it under the 'Visual C# / Silverlight' section of the 'New Project' dialog. You can download the template from here.

SilverlightNUnitProject 

To execute one or more unit tests you should use 'Run Test(s)' rather than 'Test With > Silverlight'.

RunTests

In summary you now have 3 options when testing Silverlight projects.

  1. For unit testing you can create a Silverlight NUnit Project and use the usual TestDriven.Net commands.
  2. For integration testing (within the context of a browser) you can use the Silverlight Testing Framework.
  3. For spelunking individual methods inside the CoreCLR you can use 'Test With > Silverlight'.

I'd be interested to hear how you get on.

TestDriven.Net 2.13: Support for NUnit 2.4.7

The latest release of TestDriven.Net is now compiled against NUnit 2.4.7. This version of NUnit includes Andreas Schlapsi's popular RowTest extension. The RowTest attributes have been moved to the 'nunit.framework.extensions' assembly (which you can find here: \Program Files\TestDriven.NET 2.0\NUnit\2.4\nunit.core.extensions.dll).

Here is an example RowTest method that tests a prime number function:

RowTest

The prime number function might look like this:

IsPrime

As you can see the intention of the test is very clear and there is no repetition. I've written more about parameterized unit tests here.

TestDriven.Net 2.13: Support for Silverlight 2.0 Beta 1

I've just uploaded a new version of TestDriven.Net with support for Silverlight 2.0 Beta 1. Microsoft have certainly kept me on my toes as there have been lots of changes since Silverlight 1.1. I'm sorry it has taken a while!

At the moment you're limited to running individual public methods (ad-hoc tests). If you need to run a suite of tests I recommend you use this in conjunction with the Silverlight Testing Framework that was released at MIX. Jeff Wilcox has posted a detailed tutorial that shows how to use the framework here. When running your tests using 'Test With > Silverlight', bear in mind that you're simply executing the test method and any test attributes (TestInitialize etc.) will be ignored.

TestWithSilverlight2

I've also included an application called 'agx.exe' that lets you run console applications using the Silverlight/CoreCLR from the command line. After you've installed TestDriven.Net you will find this standalone application here: \Program Files\TestDriven.NET 2.0\agx.exe. This is simply an application that I use for my own testing purposes that I thought other people might find useful.

ConsoleSilverlight2

You can download the new version of TestDriven.Net from here.

More Posts « Previous page - Next page »