TestDriven.NET by Jamie Cansdale

Fragmenting of the testing tools market?

Darrel Norton asks

Why are people creating all these new testing frameworks? Four different frameworks is at least two too many. While there might be something interesting in csUnit or MbUnit, they aren’t compelling enough for me to leave NUnit (in marketing terms these are known as switching costs). And since I’ll be using Visual Studio Team Server, I’ll very likely be using that testing framework due to all the integration work Microsoft has put into it. Why can’t we roll any functionality from the non-market leaders into the market leader (which I would assume to be NUnit, but pick one and everyone support it)? Is it pride that gets in the way? Really, the important part is to make it easier to develop better software.

I would say 4 different frameworks is 3 too many! What we need is one minimalist framework with extensibility points to allow for more advanced unit testing (say you need to run specific tests in a new app domain).

If a unit testing framework is introduced to all levels of Visual Studio, we will be doomed to at least 3 testing frameworks. One used in the SSCLI/Rotor, one used by Visual Studio (Microsoft.VisualStudio.QualityTools.UnitTestFramework) and one used by Mono and everyone else who doesn't use Visual Studio (probably NUnit)!

Testing frameworks aren't exactly rocket science. It would be possible to agree on a standard through the proper channels (i.e. the BCL/SSCLI) that everyone would use. Let's not mess it up by making 'Microsoft.VisualStudio.QualityTools.UnitTestFramework' a de facto standard!

System.Testing

Peter Provost has started a petition to include unit testing support in all versions of Visual Studio 2005.  While I agree that unit testing should be essential part of all development, I don't think Visual Studio is the place to introduce an important new API.  I think a much more fitting place would be the BCL and SSCLI/Rotor.  At the moment the BCL has an extensive set of unit tests.  I would be in favor of converting these over to using a core testing framework (say 'System.Testing').  We would then have a standard API for writing tests under all implementations of the CLI.  I think this is where real value would come.

Having said that, there is going to be a need for unit testing support in standard versions of Visual Studio 2005.  At the moment I am developing a testing tool what will work in all versions of Visual Studio and integrate with Visual Studio Team System.  It will support NUnit, csUnit, MbUnit and the testing framework that comes with VSTS.  The idea is to allow those using VSTS to run their existing unit tests unchanged vier the Team System extensibility hooks.  It will also allow those not using Team System to run tests written using the Microsoft testing framework.

So long as Team System remains a premium feature of Visual Studio, there is a point carrying on with this and other projects.  I am relieved that Microsoft have chosen to tread relatively lightly on the thriving .NET tools community.  I think this can only help encourage innovation and cross fertilization.

Visual Studio 2002, 2003 & 2005

It is getting there.  Your feedback particularly welcome if you're trying it with Visual Studio 2002 or 2005.  If it fails to load when Visual Studio first starts - check the left hand box in 'Add-in Manager' next to 'NUnit Add-In'.  Make sure it's the left hand one, you don't want it loading on startup!

NUnitAddIn & VS.NET 2002

Ronnie writes:

Having difficulty installing the Add-In into VS.NET 2002 - is it supported? The add-in appears, but when I turn it on I get 'The Add-in 'NUnit Add-In' failed to load..... Error Number: 80131534 - does that mean anything to anyone? Would upgrade, but we have too many systems to upgrade and are at a critical point in development - shame 2003's project files are not backward compatibal eh!