12-1-2004 Talk Follow-Up: Test Driven Development & .NET (Plano .NET User Group)
I want to thank everyone for taking the time out to attend 12/1/2005’s PDNUG meeting, where I presented a talk about “Test Driven Development & .NET”. I was psyched to see everyone who attended, and hoped that the session helped those who are interested in TDD in any fashion. I truly believe in this approach, and I hope that was evident during the talk.
Downloads & References:
- Slides & Demo Code – here
- My blog – http://weblogs.asp.net/scottdockendorf
- My Uber List of TDD Resources - http://weblogs.asp.net/scottdockendorf/archive/2004/10/06/238811.aspx
Please let me know if you know of some good resources missing from this list - I'll make sure to keep it updated!
Follow-Up from Q&A:
Here are some Follow-Up Items as a result of questions & comments raised during and after the talk:
#1 - Nunit & Testing Multithreaded Objects (I did not find out this person’s name)
Q: How does NUnit play/work with multithreaded objects?
A #1: NUnit - In Nunit 2.1, the Nunit.Framework.Assert method does not respond well while in a threaded environment. .NET Rock Star – Peter Provost found a way around this problem by catching an exception from on the worker thread, and pass it back to the main thread (which will report successfully to NUnit). Check out his blog post @ http://www.peterprovost.org/archive/2004/11/03/2051.aspx .
A #2: Check out MBUnit, http://mbunit.tigris.org/ , which has built-in support for multi-threaded objects.
#2 - Re: Xml Configuration Settings for TestSuites (From Rich Denis)
Q: I use the .NET System.Configuration.AppSettings class to retrieve settings in an XML file. Where do I put this logic in a Nunit suite.
A: During the session, I documented that you can place these entries in NunitGui’s app.config file, since NUNITGUI.EXE is a .NET assembly.
However – Rich Denis told me about an easier method:
“If you have a MyTestAssebmly.dll then you create a MyTestAssembly.dll config file and place it in the same folder as the test assembly. This is a bit different than I told you last night since I said to make a config file for the tested assembly not the testing assembly. I hope this helps.”
It does work – like a champ! Here is a sample C# NUnit Test with a config file. Follow these steps to verify:
- Create a C# Class Assembly
- Set a reference to Nunit.Framework.dll
- Add an app.config file, and change the file to reflect the following:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<appSettings>
<add key="MySetting" value="MyValue"/>
</appSettings>
</configuration>
- Add the following code to the Class1.cs file
using System;
using NUnit.Framework;
namespace HelloConfig
{
public sealed class ConfigReader
{
public ConfigReader() {
}
public static string GetConfigSetting(string key) {
return System.Configuration.ConfigurationSettings.AppSettings.Get(key);
}
}
[TestFixture]
public class ConfigReaderTestFixture
{
public ConfigReaderTestFixture() {
}
[Test]
public void GetConfigSettingPostiveTest() {
string key = "MySetting";
string expectedValue = "MyValue";
string actualValue = "";
actualValue = ConfigReader.GetConfigSetting(key);
Assert.IsTrue(expectedValue == actualValue,
String.Format("Values were NOT equal, expected '{0}' actual '{1}'", expectedValue, actualValue)
);
}
[Test]
public void GetConfigSettingNegativeTest() {
string key = "SettingNotThere";
string expectedValue = "MyValue";
string actualValue = "";
actualValue = ConfigReader.GetConfigSetting(key);
Assert.IsFalse(expectedValue == actualValue,
String.Format("Values were equal the keys exist. It should have NOT been present., expected '{0}' actual '{1}'", expectedValue, actualValue)
);
}
}
}
- Build the project.
- Now manually move the app.config into the bin\debug folder with the name “HelloConfig.dll.config”
- Now test with NUnit. You should find that both tests pass successfully (i.e. the values were and were not read from the config file as expected)
Thanks Rich!
#3 - MBUnit Rocks! If you want Assertions – you got ‘em! (from Rich Denis)
MBUnit is another TDD xUnit Framework, which is now part of the TestDriven.net VS.NET add-in. It provides an extremely rich Assertion model, including Arrays, collection, compiler, serialization, Security, Reflection, Text, Performance, Data, XML, web
Check out these links
- MBUnit Home Page - http://mbunit.tigris.org
- Assertions: Part 1: http://blog.dotnetwiki.org/archive/2004/05/29/274.aspx
- Assertions: Part 2: http://blog.dotnetwiki.org/archive/2004/04/29/172.aspx
- Johnathan de Halleux, Peli’s Blog - http://blog.dotnetwiki.org/
Thanks Rich!
#4 – Various TDD/Process-Related Questions (From Vince Blasberg)
Here are my takes on Vince’s questions – please add comments to this post, if you have answers
Q: Will the code be optimized at all in release mode?
A: Your test suite (class library) will be optimized (between Debug & Release builds) as normal, but nothing outside of that. As we discussed, keep in mind that your unit test code should execute in your environments, only as a test. This code should not be executed on a regular basis. Because of this, you shouldn’t have to worry about optimized code (boy that statement really sounds funny!)
This kinda leads me to a recommendation mentioned during the talk - storing all Test & Test Suites in a completely separate assembly, instead of mixing TestSuites with production code, in the same assembly. In the example of deploying to production, you could deploy all your code, execute your test (potentially in NAnt NUnit2Report task), and interrogate on the test results. If all tests pass, then you could remove your test suite assembly from production. If you encounter problems in production, you could request a change control to place the test harness back in production, and re-run your tests – to try and determine what might have caused the problem. Just a way to keep your production environment “cleansed” of obsolete or non-production code.
Q: Won’t some of the in-function tests prevent the optimizing compiler to inline some functions since they will add code and push the IL code past the 32 byte auto-inline mark?
A: Yes, it would. Good observation.
Q: Are the NUnit tests limited to just two, a one-to-one comparison and an exception expected?
A: Not at all. My demos showed just a single one-to-one comparison, and I’ll change that for the next talk. J The test structure is completely up to you, and should be driven by your functional & technical requirements, as well as security intrusion attempts. You really have no limits here.
Q: Developers must be good at adding verifications where needed. Can these unit test products perform tests in the same call to make current non-unit testing verifications part of the test? In other words, we can check for null parameters and IsLoggedIn type functions but the logical tests that must be done must be duplicated to perform unit testing. Unit tests must be reproduced every time a function is overloaded, making the test coding process work every time. Making an attribute on a function to insert and perform certain tests may help somehow and avoid repetitive test coding. Without something simpler, changed function signatures and new overloads cause more work that may not get tested unless the developer looks at a prior function and add everything as they should.
A: Good observations. Yes, if a developer adds a new overloaded method, it is their responsibility to write the associated tests for this new method.
Q: After a long night, if a developer wanted to make all green lights instead of red ones on the next morning’s build, all they have to do is comment out tests for QA to see a shippable product. ( Bad )
A: Yes, this would be very bad. I can’t deny that there still is a little human element to TDD, but isn’t’ there in many development projects. J In Extreme Programming, this should be caught due to the framework’s pair-programming model.
There are software solutions to help solve this problem (I believe). Basically, the way to solve this automatically is to place a threshold level for the code coverage results calculated during the tests. If your tests did not meet the code coverage threshold amounts, then the developer couldn’t check in the code. There are problems with that idea, because the threshold values are a moving target depending on where the developer is in the process. I know there is logic like this in VSTS, and there might be in SourceGear’s Vault (not sure at this time). I don’t know if PVCS or other popular source control applications have similar capabilities.