TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out - ISerializable - Roy Osherove's Blog

TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out

- VS 2005 is cool. You get great Intellisense and there's a even a nice refactoring that lets you create method stubs when you write code that uses a method that does not exists.
- VS 2005 also has a great feature for unit testing that allows you to create unit tests for existing classes by simply Right clicking on the classes ans selecting "Create tests".
- When you do this VS 2005 also adds another feature in the unit testing framework that creates a wrapper object for your test class. This wrapper object uses reflection to exposes any public and PRIVATE methods of your tested class as if they were public (the merits of this approach can be debated to death, no doubt. Personally, I'm against it). That means all your tests (that were generated) will use the *wrapper* object in the code, and not the actual production code.
 
Now - couple all these three features together and what do you get?
Imagine you just generated a test for your class, and you want to drive the features of this class in a test-driven manner, that is, you want to first write tests that fail, and only then write the production code to make them work.
So, you write a line of code in a new test that uses the wrapper object that was generated for you by the test framework. Suppose you want to create a method named "add" on a calculator object. your code could look like this:
 
[TestMethod]
public void TestAdd()
{
   CalculatorWrapper calc = new CalculatorWrapper();
    int result = calc.Add(1,2);
      Assert.AreEqual(3,result);
}
 
obviously this test won't compile, but! now we can use feature number 1 in the above list: with right click on the "calc.Add" line and select "implement stub". What do you expect should happen?
Yeah. That wouldn't work. Because the actually stub would be added to the *wrapper* object that was generated for you, and not to the actually class. So now you need to manually add the method to the class or not use the wrapper object at all.
Let's say you want to continue using the wrapper object, you then manually add the empty method to the class. The test will still fail! Remember that you implemented the stub automatically in the wrapper object? the default template for an empty method stub throws a System.NotimplementedException. Yes, you now need to manually added code in the wrapper object to make sure it calls the right method on your actual tested calculator object. since all that code uses reflection, that could get messy fast.
 
In short - TDD and automatically generated wrapper objects don't mix too well in the current versions of VS 2005.
Beware. You usually do TDD and should do it on the real object. Wrapper objects should stay where they belong - on tests that you generated.
 
So what happens if you DO want to write tests for private methods? well, than you can use the wrapper, but make sure you re-generate it every time you refactor your code - private methods change *all the time* - your tests will break *all the time* too.
Published Tuesday, February 22, 2005 1:42 AM by RoyOsherove
Filed under:

Comments

Tuesday, February 22, 2005 9:37 AM by Darrell

# re: TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out

Agreed. Or you could just implement the reflection yourself, it isn't that hard. That does not address the brittleness in your tests, though.
Tuesday, February 22, 2005 1:01 PM by Scott

# re: TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out

One other interesting approach I've seen with TDD is to make all the "significant" methods public so they can be exposed to the test classes, but then define separate interfaces to be the contract external callers use. This way you get easier testability by not having to go through a wrapper/reflection while at the same time still retaining some control on which methods external objects can call. In terms of "significant" methods -- they would be those that had enough complexity behind them that you feel they warrant their own test case. It also requires thinking about the interface separate from the class itself which might be considered an added benefit.
Tuesday, February 22, 2005 1:24 PM by Roy Osherove

# re: TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out

Scott: Nice idea.
Thursday, February 24, 2005 7:26 PM by Ryan Milligan (MSFT)

# re: TDD and test generation in VS 2005 can be annoying and tricky if you don't watch out

Hello everyone, I'm the dev for this feature, and I just wanted to clarify a few points on this issue.

First of all, the generated tests will only use those wrapper classes (officially, we call them "private accessors") if the class being tested is non-public, or if you selected a non-public member to test. If your class is public, then the "target" variable in the generated test method will be an instance of the type being tested. If it's not public, then we'll use the first public type in its inheritance hierarchy. If the class is public but the member we need to invoke is private, then we'll still declare target as your type, and then we'll create a separate variable to store an accessor created from the target, and invoke the member on *that*.

To sum up, if your type is public, there will always be a variable of that type in the generated test, and if you use Generate Method Stub on it, then the method will be generated in the expected place. We expect people who use private accessors in their hand-written tests to follow the same pattern. Unfortunately, it's unlikely that we'll be able to forward refactoring operations on private accessors to the code under test, or vice versa (so that if you Reorder Parameters on your code-under-test, for example, your test code that uses a private accessor will be fixed up as well). For now, we attempt to mitigate the problem by using private accessors only where necessary -- public members don't show up on private accessors (unless the type itself is private, in which case it's not really a public member to begin with), so all the refactoring features should simply work as expected for public things (even if we have to call a private constructor to create the object, we still declare the variable as the public type and only use the accessor to call the constructor itself), and you can't get cross-project refactoring on non-public members today anyway because they're not exposed to other projects, so you're not really "losing" anything by using the private accessors. That said, we would love to implement this feature, and it's definitely on the list of features to investigate for the next version.

We've also done a lot of thinking about the brittleness of private accessors. We plan to investigate a feature for this version that will result in private accessors being automatically updated as your code changes (or at least when you compile, the actual details aren't finalized yet). This means that, while refactoring still won't work for private methods (see above comment about how it doesn't work today in that situation anyway), at least you'll get meaningful compile errors when your private methods change, which you can then fix normally. The plans for this change are still tentative and we're not sure yet if it will be doable, but every customer that asks for this feature makes it more likely to happen, so please send in your vote on whether you think this would be useful.

We really appreciate the feedback. Thank you!
Saturday, February 26, 2005 12:50 PM by TrackBack

# Response to the test generation

Saturday, December 15, 2007 3:53 AM by it360

# Visual Studio2005下配置及运行NUnit

知道.net下有个NUnit,一直没有用它来写程序。今天测试了下试试,写点心得出来,一边写程序一边还得测试,浪费了很多时间精力。代码有了一定规模了,慢慢体会到单元测试的作用。用Nunit进行单元测试能及时发现新的Bug,保证原有的功能正常运行。而不必手工一个个的去试验,这是很宝贵的。在NUnit的安装目录的bin下面有两个config文件:nunit-gui.exe.config,nunit-console.exe.config,其中有一段startup的配置段,默认如下:

Sunday, December 16, 2007 9:28 PM by csdnexpert

# Visual Studio2005下配置及运行NUnit

知道.net下有个NUnit,一直没有用它来写程序。今天测试了下试试,写点心得出来,一边写程序一边还得测试,浪费了很多时间精力。代码有了一定规模了,慢慢体会到单元测试的作用。用Nunit进行单元测试能...