TestDriven.Net 3.0 – All Systems Go

I’m pleased to announce that TestDriven.Net 3.0 is now available. Finally! I know many of you will already be using the Beta and RC versions, but if you look at the release notes you’ll see there’s been many refinements since then, so I highly recommend you install the RTM version. Here is a quick summary of a few new features:

Visual Studio 2010 supports targeting multiple versions of the .NET framework (multi-targeting). This means you can easily upgrade your Visual Studio 2005/2008 solutions without necessarily converting them to use .NET 4.0. TestDriven.Net will execute your tests using the .NET version your test project is targeting (see ‘Properties > Application > Target framework’).

TargetFramework

There is now first class support for MSTest when using Visual Studio 2008 & 2010. Previous versions of TestDriven.Net had support for a limited number of MSTest attributes. This version supports virtually all MSTest unit testing related attributes, including support for deployment item and data driven test attributes. You should also find this test runner is quick. ;)

MSTest

There is a new ‘Go To Test/Code’ command on the code context menu. You can think of this as Ctrl-Tab for test driven developers; it will quickly flip back and forth between your tests and code under test. I recommend assigning a keyboard shortcut to the ‘TestDriven.NET.GoToTestOrCode’ command.

GoToTest

NCover can now be used for code coverage on .NET 4.0. This is only officially supported since NCover 3.2 (your mileage may vary if you’re using the 1.5.8 version).

TestWithNCover

Rather than clutter the ‘Output’ window, ignored or skipped tests will be placed on the ‘Task List’. You can double-click on these items to navigate to the offending test (or assign a keyboard shortcut to ‘View.NextTask’).

TaskList

If you’re using a Team, Premium or Ultimate edition of Visual Studio 2005-2010, a new ‘Test With > Performance’ command will be available. This command will perform instrumented performance profiling on your target code.

Performance

A particular focus of this version has been to make it more keyboard friendly. Here’s a list of commands you will probably want to assign keyboard shortcuts to:

Name Default What I use
TestDriven.NET.RunTests Run tests in context   Alt + T
TestDriven.NET.RerunTests Repeat test run   Alt + R
TestDriven.NET.GoToTestOrCode Flip between tests and code   Alt + G
TestDriven.NET.Debugger Run tests with debugger   Alt + D
View.Output Show the ‘Output’ window Ctrl+ Alt + O  
Edit.BreakLine Edit code in stack trace Enter  
View.NextError Jump to next failed test Ctrl + Shift + F12  
View.NextTask Jump to next skipped test   Alt + S

 

By default the ‘Output’ window will automatically activate when there is test output or a failed test (this is an option). The cursor will be positioned on the stack trace of the last failed test, ready for you to hit ‘Enter’ to jump to the fail point or ‘Esc’ to return to your source (assuming your ‘Output’ window is set to auto-hide).  If your ‘Output’ window isn’t set to auto-hide, you’ll need to hit ‘Ctrl + Alt + O’ then ‘Enter’. Alternatively you can use ‘Ctrl + Shift + F12’ (View.NextError) to navigate between all failed tests.

Output 

For more frequent updates or to give feedback, you can find me on twitter here.

I hope you enjoy this version. Let me know how you get on. :)

TestDriven.Net 2.24: XCopy Deployable Test Runners

There is a new feature in TestDriven.Net 2.24 (and 3.0 Alpha) that should greatly simplify the deployment and versioning of test runner plug-ins. In previous versions it was important that a compatible test runner plug-in was installed for each test framework you use.

This was manageable if all of your projects used the same version of a test framework – however it could quickly get out of hand if a project referenced a different framework version. Because it isn’t always feasible to upgrade all test projects at the same time, teams could end up trapped using an old version of a test framework or with unit test projects that are difficult to run.

As of TesDriven.Net 2.24, it is now possible to include the test runner alongside its test framework assembly. The advantage of doing this, is your tests can be executed without any installer dependencies! For example, you could use any version of xUnit, MbUnit or NUnit and your teammates would be able to execute the tests without needing to worry about which version of xUnit or Gallio to install!

This feature is enabled using a file with a .tdnet extension, which lives in the same directory as your referenced test framework assembly (e.g. ‘mbunit.dll’, ‘xunit.dll’, ‘nunit.framework.dll’ etc.). If you’re using Gallio 3.1 or xUnit.net 1.5, you will find the required ‘MbUnit.dll.tdnet’ or ‘xunit.dll.tdnet’ file already exists. For earlier versions of Gallio, xUnit or MbUnit – this feature can be enabled by crating a simple XML file.

For example, say you have a test project that uses MbUnit 2. This test framework was released long before .tdnet files existed. To make the MbUnit test runner xcopy deployable, you would need to create a ‘MbUnit.Framework.dll.tdnet’ file and copy the contents of ‘\Program Files\MbUnit\bin’ into version control.

The ‘MbUnit.Framework.dll.tdnet’ looks like this:

<TestRunner>
       <FriendlyName>MbUnit {0}.{1}.{2}</FriendlyName>
       <AssemblyPath>MbUnit.AddIn.dll</AssemblyPath>
       <TypeName>MbUnit.AddIn.MbUnitTestRunner</TypeName>
</TestRunner>

The ‘MbUnit.Framework.dll.tdnet’ file should be placed alongside ‘MbUnit.Framework.dll’, ‘MbUnit.AddIn.dll’ and the rest of MbUnit 2. Once you have referenced the ‘MbUnit.Framework.dll’ assembly in this directory, you can uninstall MbUnit 2 and still use TestDriven.Net to execute your tests!

If you’re using a version of xUnit.net prior to 1.5, the ‘xunit.dll.tdnet’ file would look like this:

<TestRunner>
  <FriendlyName>xUnit.net {0}.{1} build {3}</FriendlyName>
  <AssemblyPath>xunit.runner.tdnet.dll</AssemblyPath>
  <TypeName>Xunit.Runner.TdNet.TdNetRunner</TypeName>
</TestRunner>

If you need help making this work with any other test framework, please let me know and I’d be happy to post the details here.

I hope you enjoy the simplified deployment and versioning of whatever test runner you use!

You can download TestDriven.Net 2.24 & 3.0 Alpha from here.

TestDriven.Net 3.0 Alpha – Support for Visual Studio 2010 Beta 2

For the best part of a week, I’ve been battling with the Visual Studio 2010 Beta 2 extensibility model. I’m happy to report that - finally - I’ve managed to make it bend to my will. ;-)

 vs2010b2

At this point I’ve decided to fork TestDriven.Net 2.x and 3.x. Going forward, TestDriven.Net 3 will support Visual Studio 2010, 2008 & 2005. TestDriven.Net 2 will continue to support Visual Studio 2008, 2005 & 2003. They will be developed in parallel until the RTM versions of Visual Studio 2010 and TestDriven.Net 3.0 are released. I will then focus my energy on TestDriven.Net 3.x.

With TestDriven.Net 3.0 Alpha, you can expect the Visual Studio 2008 & 2005 support to be stable.

In order to support Visual Studio 2008 Beta 2, I’ve needed to do a number of pretty gruesome workarounds. One of these workarounds involves repainting TestDriven.Net’s command bar button icons each time Visual Studio starts (Beta 2 doesn’t persist them). All previous versions of TestDriven.Net have activated themselves on demand rather than on startup (when Visual Studio loads). This meant installing TestDriven.Net had no impact on Visual Studio startup times.

If you’re sensitive to Visual Studio startup times, it’s still possible to make TestDriven.Net activate on demand in Visual Studio 2010. Simply uncheck the ‘Startup’ box next to TestDriven.Net in the ‘Add-in Manager’. Be warned, however, that TestDriven.Net’s command bar buttons pictures will appear blank until the first time they’re used.

startup2010

For the ‘Test With > Coverage’ functionality to work with .NET 4.0, you’ll need to install NCover 3.2 or higher (see here). You can also use ‘Test With > Team Coverage’ if you’re running Visual Studio 2010 Premium or higher.

team2010

You can download TestDriven.Net 3.0 Alpha from here.

Which test runner are you using ?

Fabio Maulo is doing a poll to find out which test runners people are using:

poll

You can take the poll and see the results here.

Posted by Jamie Cansdale | 19 comment(s)
Filed under:

TestDriven.Net 2.23 Beta – Cleaner Stack Traces

You have probably noticed with previous versions of TestDriven.Net, stack traces could turn into an unruly mess of file path information.

trace1

In TestDriven.Net 2.23, source file paths are displayed as relative to their containing project.

 trace2

There are a few cases where using a project relative path would be ambiguous or a source file outside your solution is referenced; in these cases an absolute file path will be used. In general however, you should find that stack traces are much cleaner.

Here are links to download read the release notes for TestDriven.Net 2.23 Beta. Please contact me if you find any issues with this version.

TestDriven.Net 2.23 Beta – Faster!

If you’re using TestDriven.Net 2.23 on a 64-bit OS, you should find that running tests is almost twice as fast!

It turns out that the JIT compiler for x64 is considerably slower than for x86 (and uses more memory). You can easily improve the startup time for your EXE projects by changing them to compile for x86. In fact this is what Microsoft is doing in Visual Studio 2010 (the default for EXEs will be x86). It’s only worth compiling EXEs for AnyCPU if your application might use a lot of memory. See Rick Byers’ post on the subject: AnyCPU Exes are usually more trouble than they're worth.

run64

run32

For this reason the default for AnyCPU test projects is now to execute in a 32-bit (WOW) process. If you need to temporarily run tests in a 64-bit process, there’s an option to change the default.

AnyCpuOption

If you have tests that check behavior in a 64-bit process, you can add them to a test project that compiles for the x64 platform.

platform_target

Because this is a significant change, I thought it warranted its own post. There’s a lot more in this release that I’m planning to blog about later.

Here are the download and release notes for TestDriven.Net 2.23 Beta.

TestDriven.Net 2.22 RTM - What’s New?

I'm happy to report that TestDriven.NET 2.22 RTM has cleared the launch pad!

It has been a while since the previous RTM version, so here is a quick recap of what’s new:

NUnit 2.5

TestDriven.Net now includes the production release of NUnit 2.5. This is the recommended version of NUnit if you’re using .NET 2.0 or above. To ensure maximum compatibility with .NET 1.x and legacy NUnit extensions, the last releases of NUnit 2.2 & 2.4 are also included.

nuni25

(more here)

64-bit Windows

By default TestDriven.Net will execute test projects compiled for AnyCPU in a 64-bit process. There are however some test runners that must run inside a 32-bit process. If you are testing with NCover 1.5, Team Coverage, dotTrace or .NET 1.1 (MSBee), TestDriven.Net will automatically fall back to using a 32-bit process. To force your tests to always execute in a 32-bit process, you can specify ‘Platform target: x86’ in your test project’s build properties.

x86

F#

F# is now a first class citizen inside TestDriven.Net. To enable stack traces on failed assertions, you must specify ‘Other flags: --optimize+ notailcalls’ on your test project’s build properties. For best results use NUnit 2.5, MbUnit 3.0 or xUnit 1.1 (which all support test attributes on static methods).

fsharp

(more here)

Options

A new ‘TestDriven.Net’ options pane is available when using Visual Studio 2005 and above. This can be used to selectively include/exclude tests in specified categories, choose when to display trace/debug output and choose to use new test process for each test run (as a workaround for badly behaved tests).

options

(more here)

Visual Studio 2010 Beta1

Previous beta versions of TestDriven.Net were compatible with the PDC 2008 release of VS 2010, but not the VS 2010 Beta1 release.  To enable support for VS 2010 Beta1, you must do a ‘Complete’ install and assign keyboard shortcuts to the TestDriven.Net commands you wish to use.

vs2010 

(more here)

Typemock Isolator

TestDriven.Net now includes active support for Typemock Isolator. You can disable the Typemock Isolator add-in and leave TestDriven.Net to enable Isolator for test projects that require it. This can significantly improve performance when starting applications/web sites from Visual Studio and executing tests that don’t require Isolator.

typemock

NCover

This version of TestDriven.Net has been tuned to work with NCover 1.5.8 (included with TD.Net) and the commercial NCover 2.x & 3.x versions. When you ‘Test With > Coverage’, the contents of NCoverExplorer will be automatically refreshed and only classes with debug symbols will appear in the tree view.

ncover

(more here)

You can download the latest version from here. If you’re a professional software developer, perhaps your employer would be interested some Enterprise licenses. Feel free to point the appropriate person in your organization to the licensing & purchase page or contact us for a quote. :)

I’m now going to start working on the next version in earnest. You are welcome to contact me with any suggestions you have for a future release.

Thanks for your support, bug reports and interest!

TestDriven.Net 2.22: Support for Visual Studio 2010 Beta 1

I’ve just uploaded a new version of TestDriven.Net (2.22 RTM) which is compatible with Visual Studio 2010 Beta 1.

Unfortunately this support is somewhat limited because command bar extensibility has been disabled in VS 2010 Beta 1. All is not lost however because the commands still work when executed using a shortcut key. This is how many power users invoke TestDriven.Net already and can make for even less friction when executing your tests.

To enable support for VS 2010 Beta1 you will need to select ‘Complete’ when installing TestDriven.Net. (Support for VS 2010 currently disabled when doing a ‘Typical’ install.)

setup

You can use the ‘Environment > Keyboard’ pane on the ‘Options…’ dialog to assign new shortcut keys. To list all of TestDriven.Net’s commands, simply: “Show commands containing: testdriven”.

You will probably want to assign shortcuts for the following commands:

TestDriven.NET.RunTests: Run Test(s)
TestDriven.NET.Debugger: Test With > Debugger
TestDriven.NET.RerunTests: Repeat Test Run
TestDriven.NET.Reflector: Go To Reflector
TestDriven.NET.ApplicationNUnit25: Test With > NUnit 2.5 (start the NUnit GUI)

shortcut

Alternatively I’ve created a .vssettings file you can import using ‘Tools > Import and Export Settings…’.

The keyboard assignments are as follows:

Alt+T: Run Test(s)
Alt+R: Repeat Test Run
Alt+D: Test With Debugger
Alt+E: Go To Reflector
Alt+N: Launch NUnit 2.5
Alt+V: Test Inside VS
Alt+B: Build Current Project

Once imported I’m using Alt+T to execute an ‘Ad hoc’ test from inside the ‘go’ method.

test1 

You can download TestDriven.Net 2.22 RTM from here.

TestDriven.Net 2.21: Now includes NUnit 2.5 RC

Support for NUnit 2.5 RC

A couple of days ago, Charlie Poole announced that the first NUnit 2.5 release candidate is now uploaded. I’ve hurried to get a new version of TestDriven.Net with support for this version ready. I’m happy to announce that TestDriven.Net 2.21 Beta (which includes NUnit 2.5 RC) is now available for download.

TestWithNUnit25

This is a major NUnit release with lots of new features. Here is the list that was posted on the NUnit mailing list:

  1. Parameterized (data-driven) tests are supported, with features similar to those found in mbUnit and xUnit.net.
  2. Theories - as used in JUnit- are supported fully, including support for Assume.That.
  3. New attributes allow the specifying the thread and apartment state requirements of a test.
  4. Exception handling can now be moved into the test code using Assert.Throws or the Throws.Exception syntax.
  5. Test methods and fixtures may now be generic and many asserts and constraint expressions now support generic syntax.
  6. Many constraints now permit substitution of a user-defined comparison algorithm through the Using modifier. Lambda expressions are supported.
  7. Test execution may now take place in a separate process for better isolation.
  8. Tests may be loaded and executed using a selected runtime version.
  9. Tests, setup methods, teardown methods and data sources may be static if desired. If there are no instance methods, then NUnit doesn't need to construct your test class.
  10. Source code is displayed in the gui, where available.

Charlie has also posted a top 10 reasons try NUnit 2.5.

Improved x86 / 32-bit support on a 64-bit OS

This version also contains improved support for running tests in a 32-bit process. To configure a test project for 32-bit execution, simply select ‘x86’ as the ‘Platform target’ in the project properties. After doing this, the ‘Run Test(s)’ and ‘Test With > NUnit 2.5’ commands will execute your tests in a 32-bit process (this doesn’t work for NUnit 2.4). By default projects configured for ‘Any CPU’ will execute tests in a 64-bit process (unless you’re testing with NCover, Team Coverage or dotTrace).

x86  

Experimental Visual Studio 2010 support

TestDriven.Net was working with the Visual Studio 2010 PDC CTP version. Unfortunately I’ve had less luck with more recent pre-Beta versions (Visual Studio was crashing). You can try the experimental support by choosing ‘Custom’ setup and enabling the ‘Visual Studio 2010’ feature. There is a chance this will be working again when the Visual Studio 2010 Beta is released.

vs2010

Trace/debug messages now appear in ‘Debug’ pane

When stepping through code started from TestDriven.Net, you may have noticed that trace/debug messages appeared in the ‘Test’ pane but not the ‘Debug’ pane. Trace/debug diagnostic messages will only appear in the ‘Test’ pane when targeting specific tests, but not when executing all tests in a project / solution.

Release Notes & Download

Here are links to the TestDriven.Net release notes and download.

Don’t hesitate to contact me or the NUnit team if you find any issues with this release.

TestDriven.Net 2.20: Improved NCover Integration

In the latest release of TestDriven.Net, you’ll find much improved integration with all versions of NCover:

Support for 64-bit Windows

‘Test With > Coverage’ now works on 64-bit versions of Windows. In previous versions this could be made to work by compiling your test project for x86. The new version will automatically execute your tests in a 32-bit process if a 64-bit version of NCover can’t be found. (There is similar 64-bit Windows support for dotTrace and Team Coverage)

No more ‘Reload coverage file?’ dialog

NCoverExplorer 2.1 & 3.0 will no longer prompt to reload the coverage file. The coverage report will automatically refresh when new data is available. If you’re using NCover 3.0.18 or higher, you won’t be prompted to save a ‘New Project’ when you close NCoverExplorer.

Automatic assembly filtering

If you’re using NCover 3.0, only assemblies that have a corresponding PDB file will appear in the ‘Explorer’ window. I’m hoping this is a good alternative to explicitly naming assemblies that shouldn’t appear in the coverage support.

Support for Typemock Isolator

You can now execute ‘Test With > Coverage’ with NCover 3.0 and Typemock Isolator installed. (Previous versions would fail with the error: ‘Couldn't find TypeMock profiler name for NCover 3.0’.)

Recommended Versions of NCover & Typemock

If you have a commercial license for NCover, make sure you’re using NCover 3.0.18 or later. If you’re using Typemock Isolator, I recommend you use at least version 5.3.0.

 

If you’d like more timely updates on new releases, you can find me on twitter here.

More Posts Next page »