MbUnit vs. NUnit Vs. Team System Unit Testing - Choosing a unit test framework
Update: There's a new Unit test framework called XUnit.NET. More info here.
MbUnit is gaining some momentum as one of the nicest unit testing frameworks for .NET out there. I just came out as an RC2 release with many new features an enhancements, including dull support for NUnit 2.4 Asserts - which is a big deal.
The two coolest features that draw people to using it are:
- [Rollback] attribute that automatically rolls back any changes to the database that were made during that test method (using a method I developed with COM+ 1.5)
- [RowTest] attribute that allows declaring one test with several inputs (acting as several tests).
I also like MbUnit's extensibility model with is largely different than that of NUnit's and is easier to conform to when writing new attributes.
MbUnit is also one of the topics that I continually have a dilemma about using. Since its not the de-facto standard out there, I always feel there's a chance of it being frozen in mid development, making all those who depend on it frozen along with it.
[Update: Andrew, one of the leads on mbUnit, makes some good points on why he thinks this is far from happening - which is reassuring, and I'd like to hold out longer befomre making a final call on it's "frozeability".]
I still recommend people to use NUnit as the standard. You can do a lot with NUnit (as well as the rollback attribute - See XtUnit for that). When you get used to it, then I suggest you check out MbUnit. you'll have the tools to whether you really can't live without it.
As long as there are two primary contenders in the unit testing world for .NET (NUnit and MbUnit - don't even get me started on Team System Unit Tests I discuss them in a bit) - there will always be that dilemma and the situation will always be a bit confusing for newcomers into this world.
If your heart is set on using MbUnit (and it is a good platform - I'd use it in a heart beat if a client was using it that I was consulting to) - I recommend using TestDriven.NET along with the unit tests. It just makes it much easier (plus you don't have to worry about any custom UI to run them in - it runs NUnit and MbUnit tests with no problems).
What about Team System unit testing?
For now the unit tests framework in team system and VS 2005 cannot be the de-facto standard for several reasons:
- It sucks too much right now. They have waay too many bugs in them (excepted exception attribute especially broken
- It doesn't have enough features and asserts to support the most common cases
- It's not free for everyone to use ( you can't even use it with the VS express editions - shame on you!)
- It requires VS 2005 to run (no separate run-time UI - although there is a command line)
A good unit testing solution should solve at least these four basic points in order to be considered as a candidate for a de-facto framework. Not even mentioning wide user acceptance : team system does not live up to its hype right now.
The only good thing going to it is that if you use Team Foundation server you can run the unit tests you wrote more easily as part of the automated build - which you can also very easily achieve if you work with any other framework such as NUnit by using a custom build action. While out of the box experience is important, and should not be dismissed easily, I do tend to still use NUnit even with Team System - as I tend to get better features, more stability and a wider range of possibilities for running my unit tests that way.