Reply to Scott Bellware's Review of my Unit Testing with Whidbey article

Scott Bellware, INETA Speaker Chair, Microsoft MVP, and Austin .NET User Group President recently posted a review of my .NET Developer’s Journal Article, “Unit Testing with Whidbey”.  

• The article is located here:  (Web-based article,  Entire Issue PDF)
• Images are located here
• His review is located here. 
• My response is this page :)
• Update: His response to my response is here:

Quite honestly, I truly thought that (a) mentioning TDD in a “Unit Testing with Whidbey” and/or (b) promoting a interface first approach to TDD would cause a stir among those who following the traditional TDD approach (create tests before ever writing a line of TDD code).  I just wanted to go on the record and help shed light on the article, why I wrote it the way I did, and my intended audience.

My response (as posted in his blog here) is as follows:

<< In reading and re-reading the article, I'm left feeling quite unsure whether I know where Scott is coming from, and I find it hard to distinguish when he's talking about plain old unit testing and Test-Driven Development.It's a good overview of VS 2005's support for unit testing, and it harkens the coming focus that will be given to unit testing in the next generation of .NET development.>>

Hey Scott,

After talking with Chris Menegay, I had a feeling that it would strike a cord with you. J  Seriously, I appreciate your input, and you made many valid points.  I wanted to help shed some light on the article, and why I wrote it the way I did:

<< In some cases, I found the TDD moniker used in context in a way that was misleading and adding to TDD misconceptions.  I fear that I might just be reading the article from the wrong perspective and not seeing some of the points being made, but I fear even more that others might misread the article as well, and make their first forays into TDD on shaky conceptual footing. >>

The focus of this article was to highlight the new Unit Testing Features of Whidbey (Visual Studio 2005).  It was not intended to be a comprehensive review of TDD, nor mention current open-source applications available today, including TestDriven.net, NUnit, csUnit.  Also, I only had 1,500-2,000 words to work with, which forced me to summarize where necessary.  I had to remove almost all my Refactoring content due to this reason (yes, it made me sad too).

Why Mention TDD Approach?

There were 2 reasons that I mentioned TDD in the article.  First, I too (along with you) am fond of the TDD methodology.  I’ve been using it for a few years now (using NUnit), and love it.  Through my “giving back to the community” interests,  I wanted to introduce TDD to those who are new to the methodology.  To me, it makes sense to at least mention TDD when it pertains to Unit Testing.   

For the second reason, let’s compare and contrast the two common TDD approaches

Now, let’s look at the “selling points” for using Whidbey as your Unit Testing engine (at least with Beta 1 Refresh as I used for the article)

• Integrated UI (Test Management, Categories, UI to select individual tests) – These are nice features, really independent of which TDD approach you use
• Code Generation (from Object Interfaces)– This is impossible to “sell the concept” to individuals following the test/code/refactor approach, because you have no object interfaces to generate code from – the tests come first.  
• Integrated Access To Code Coverage – This is a nice feature, but independent of which TDD approach you use.

Thus - you might agree that “Code Generation” isn’t a benefit for Approach #1 – Test First Developers, because the interfaces come from the tests, not from the project design (i.e. you don’t have an object interface to generate code from).  Thus, I couldn’t write an article based on the Test-First approach.  It is, however a big benefit for people who follow Approach #2, which is why I chose to use that approach in the article.  

My personal beliefs is that TDD can benefit any development environment using any approach.   I know there are those who only recognize Approach #1, and I respect that.  My experience in consulting has led me to believe that any approach/methodology needs to be flexible, support team development, and conducive to change.  IMO (which is all it really is), Approach #2 offers that.   

During my TDD talks that I’ve presented at our local .NET User Groups, I was always asked with the same concern = “all that extra code!”  I agree that this is a misperception, and wanted to discuss my ideas on that topic.  I thought that people who follow Approach #2 might benefit greatly from Code Generation, which is why I decided to include it in the article.  

For those who don’t recognize Approach #2 as a valid methodlogy, I completely understand why the article might be confusing.  I stated at the beginning of the article that  the article focuses on Approach #2, and not Approach #1.  However, based on your review, I don’t think I made the point clear enough.    (In fact, I had to remove the comparison table for space considerations).  Thank you for that, and I will clarify that stance in further articles and presentations.

<< 1. Thoroughly design your software. Know exactly what the software must accomplish before writing a single test.
Point number one gave me pause.  It may just be the way Scott chose to word this point, but it can be taken to mean something that couldn’t be more at odds with TDD and XP.  TDD process helps achieve good design (good and simple is the goal), but it doesn’t necessarilly start with good design, and certainly not often with thorough design.>>

You bring up a great point in Approach #1 vs. Approach #2.  Using Approach #1, the design starts and ends with your TDD cycle.  Using this approach, you have a basic idea on how you want your code to perform, and use the multiple test iterations to refine your design.  Environments that use Approach #2, tend to design their projects up front, possibly because those shops follow traditional waterfall project management and need to show a design before obtaining the resources to staff the project.  In this case, the design could be close to completion before development begins.

Why do I feel this way?  My consulting experiences include many different types of industries and different functioning development shops.  I’m not bragging, but just to drive home the point that, only a few of these shops use Extreme Programming – many use traditional waterfall approaches – which require LOTS of documentation and planning before development begins.  In addition, from a consulting standpoint, especially during the .com bust, many of our clients would have not supported XP (dual programming model, lack of documentation, etc..).   IMO – TDD still works for these shops, just using Approach #2.  

<<That said, having solid requirements and writing low-level stories is good and desired, so I'm confused.  I don’t know if Scott is saying do detailed design up front, or have good requirements up front. >>

Very good assessment on my choice of wording.  I should have mentioned solid requirements in addition to design.  

<<In point three, Scott confuses me with the use of the plural "tests" in "Use design requirements to build and execute tests", and the use of "them" in "and watch them fail". >>

I should have asked you to tech review the article before going to print J  Another great point, that could present a grey cloud above those new to TDD.  This statement should have been reworded to talk about executing the process for each individual test.

<<In point number five, Scott seems to suggest that code tweaks and refactoring are one and the same.  I not sure that I would personally wholeheartedly agree with the equivocation of refactoring and tweaking for performance or even structure.  Although Scott references Fowler's refactoring catalog, so I guess he's just kinda using some jargon when making this point.  And in truth, I guess it's the jargonizing of agile approaches that I'm trying to shed some light on.  In this particular case, I would want to stand up (and I guess I am) and say that tweaking and refactoring are not equivalent.>>

LOL – no I didn’t throw in Fowler reference to give the paragraph validity.  As mentioned earlier, I had to cut out all my refactoring examples due to space constraints. And I couldn’t agree more – tweaking & refactoring are not one in the same.  Tweaking is just that, resolving small issues.  Refactoring is much more involved and could be a whole article/book in itself.  My initial cut was 3,600 words (way over the limit), and since I committed the article to focus on the new features of the Whidbey Unit Testing engine, I had to cut it out of the article.  

<< Test-Driven Development is a design discipline.  TDD is a rigorous programming methodology that brings the levels of formalism previously only found in OO analysis and design methods down to the level of designing and constructing code and into the hands of the programmer.  Unit tests are a side effect of TDD. If you're generating tests from pre-coded classes, chances are you're not doing TDD.  If you're generating tests from pre-coded classes, and you've never really done any unit testing before, and you assume that you're doing TDD, chances are that TDD is going to go sailing right over your head, maybe never to be seen or heard from again. >>

Another great point, and something that shouldn’t have been cut out of the article.  I had 3 or so paragraphs on this point, stating that simply having LoadTest & SaveTest methods on an object that has Load & Save methods is not sufficient.  As a TDD developer, you MUST think outside the box, figure out every conceivable way that your object will be called, used, destroyed, etc…  You will end up with MANY more tests than methods on your object – if not, then you’re doing something wrong.  Take for example a Load method on a CRUD object.  You’ll want (at the bare minimum) tests for

• Postive Scalar Load
• Negative Scalar Load
• Postive Multiple Load
• Negative Multiple Load
• Security Tests (to test for SQL Injection, etc..)
• TearDown Tests (if the objects plan to do any COM interaction, or have Dispose logic that you want to test)

While we might “agree to disagree” on the benefits on Approach #2, I hope this post helps clarify my approach, and that we’re not really that far off, from a TDD evangelism standpoint.  

I appreciate your honest review.