Issues in breaking dependencies for unit testing
One of the most important issues I've come to realize is the various ways of dealing with legacy code, mainly - how to deal with breaking multiple dependencies on an existing class or method that needs to be tested.
There are two basic ways to break dependencies, with various "dialects" for each one. The both have their pros and cons and I won't explain in detail how to do them, but mainly, which of them you should choose and when.
Mock Objects
The first method of breaking dependencies is that of "Mock Objects".
In short, it relates to how one would send an instance of a "fake" class to be interacted with from the class under test instead of the *real* class. For example. to test a method that calls various Logging methods against a class that logs stuff to the database, you might be inclined to replace the instance of the logging class with that of a fake one, so that any calls made to the log will not actually go to a database, and thus will run much faster and won't need any configurations before the tests are run. That fake class can also tell us whether the "Log" method was called or not and how many times, since it is totally under our tester's control (they created it).
This article will deal mainly with the second main approach to testability, so if you’d like to read more about the Mock Objects method, there is plenty of material available on this over at www.MockObjects.com (also called the “Dependency Inversion” or “Inversion of Control” pattern). Martin Fowler also writes about this pattern in this fine article, though it is less related to testing and more generic in nature.
Extract & Override
The second method of breaking dependencies is much closer to the tested code, and relies heavily on inheritance.
It is usually called "Extract & Override".
The main idea here is that instead of "faking" the objects which are problematic. you "fake" the actual calls to them. An example might be in order.
Suppose we have a class called "MyCalculator" which contains code like this:
public double Calculate(int x, int y)
{
if(x>1000)
{
Logger.StaticLog("Calc":Got an illegal number");//problematic call
throw new Exception("Illegal number");
}
}
suppose we want to test this method. And suppose Logger.StaticLog() takes 10 seconds to do its magic (you'd probably do that asynchronous but this example is good enough). Obviously any tests that run this method will take too long to run, and unfortunately, this is a static method, so we can't easily "inject" or "replace" the Logger with a fake instance since there is no instance to work with. So instead, we will refactor our code to make it testable like this:
public double Calculate(int x, int y)
{
if(x>1000)
{
callLog("Calc":Got an illegal number");//refactored
throw new Exception("Illegal number");
}
}
protected virtual void callLog(string text)
{
Logger.StaticLog(text);
}
the idea is that in our tests, we won't really use this original calc class, but instead inherit from it and override the "callLog" method to not really do anything (which means you'll want to make those overridable methods with as little logic as possible) like this
[Test]
[ExpectedException(typeof(Exception))]
public void Calculate_IllegalNumber_ThrowsException()
{
TestableCalc c = new TestableCalc();
c.Calculate(1001, 0);
}
class TestableCalc: Calculator
{
public string callLogText = string.Empty;
protected override void callLog(string text)
{
callLogtext = text;
}
}
Notice that the second method allows you to break even very hard dependencies in a relatively easy manner.
Pros and Cons of Mock Objects
Pros
- Most mock object frameworks allow you test test interaction between objects in a very easy manner, by allowing you to add any number of “expectations” and return values to your mock objects that will be tested against. The syntax is clear and all of the hard work of maintaining the state of the cals to the mock object is performed with the mock object system.
- The last sentence means that tests involving a single mock object will be very readable and easy to write, especially when multiple calls are expected against it, or when parameter values are being tests for method calls (for example, to make sure that a Logger class is called 3 times with different values each time would be very easy to write with a framework such as NMock or NUnit.Mocks)
Cons
- the more depdencies you need to break on the tested object (legacy systems are usually just the thing that might have that) the mock Mock objects and stub objects you’ll need to create and initialize in your tests, and the more refactoring you will have to do to the system to make it testable (adding new interfaces, constructors, parameters and such..). this may prove almost impossible on some systems where you don’t own all the code and where refactoring needs to be done on code that you don’t own.
- The more dependencies means also less readable tests because all the initialization is not readable at all when its 20 lines of code or more before the actual test takes place. Even if you refactor your mock initialization code it might still not be very readable.
- Mock objects don’t help when you try to deal with code that calls static methods, singletons and other types of calls in which an object instance cannot be replaced (there are ways to refactor singletons into testability but you might not have that luxury)
Pros & Cons od Extract & Override
Pros
- It’s much faster and easier to refactor the system into testability with Extract& Override (perhaps this should be named “Extractoring” ? ) You don’t need to introduce new interfaces into a system and the only requirements are that the class is overridable and that the methods are overridable (protected)
- It’s faster and easier to write tests for such a system because test setup and object initialization is easier and clearer.
o To mimic system behavior override a method and return what you want using a publicly exposed member on the inheritor class or throw an exception by checking a public flag
o To check interaction calls simply check a publicly exposed member (“wasMethodXXCalled” or “GetMethodXXPassedValues()” on the inheritor class
- You can break any dependency you like right at the source.
o You can override a Boolean validation check such as if(configurationSettings[“key”]==value) into if(hasKeyValue(value))
o You can override full calls for static methods such as turning LogClass.StaticLog(“text”) into callLog(“text”)
Cons
- You have to do all the grunt work of verifying calls to methods, including parameter values, yourself. When we used mock objects, the mock object framework we used did that for us automatically. But when we do this manually (i.e inherit our own class and override methods) we need to make the code work on our own which can lead to long code inside the overriding class.
- If you need to override some methods and not others in a dynamic way (i.e different tests override different methods) you need to either implement base-forwariding logic into the overridden methods or create multiple inheriting classes which override different methods. In either case, it can get out of hand quickly.
- In some types of applications, especially Real-Time, where performance matters a lot, adding many virtual method calls may actually be a hindrance. There are ways around that though, using various conditional-compilation techniques which are not discussed in this article.
Hidden bugs with E&O – biggest hazard of all
- When you override a method call, you usually end up with at least (hopefully only) one line of code in the actually virtual base method which simply calls some external dependency and may return a value. There is a problem here that may not be recognizable at first. To explain the problem I’ll need a little example.
Here’s the code from the refactored class in the first example:
public double Calculate(int x, int y)
{
if(x>1000)
{
callLog("Calc":Got an illegal number");//refactored
throw new Exception("Illegal number");
}
}
protected virtual void callLog(string text)
{
Logger.StaticLog(text);
}
Now, consider what happens if suddenly, the design changes a bit and the logger takes in another parameter (and designs change with time). The new signature for the Logger.StaticLog now looks like this:
Public static void StaticLog(string text)
Public static void StaticLog(int severity, string text)
There is a new parameter for severity of type int, and this is a new overload for the previous method. Fine and dandy. Now suppose we want to change our code to send a severity each time we call the log instead of using the old method. We might refactor the virtual method to take another parameter such as :
protected virtual void callLog(int severity,string text)
{
Logger.StaticLog(text);
}
But notice that we don’t actually change the actual call to the static log method. In fact, in our test we would do the same thing as in the first example – override the virtual method and make sure it is called with the correct parameters. So our tests might look like this:
[Test]
public void Calculate_IllegalNumber_LogsSeverity()
{
TestableCalc c = new TestableCalc();
c.Calculate(1, 0);
Assert.AreEqual(100, c.callLogSeverity);
Assert.AreEqual(“illegal number”, c.callLogText);
}
class TestableCalc: Calculator
{
public string callLogText = string.Empty;
public int callLogSeverity = 0;
protected override void callLog(int severity,string text)
{
callLogtext = text;
callLogSeverity = severity
}
}
We’re overridng the callLog method to make sure it is called with the correct values.
To make this test pass our code might look like this:
//the method we are testing
public double Calculate(int x, int y)
{
if(x>1000)
{
callLog(100,"illegal number");
throw new Exception("Illegal number");
}
}
protected virtual void callLog(int severity,string text)
{
//notice that there is a bug here!!
Logger.StaticLog(text);
}
Yes – this example will actually pass the callLog test, even though there is an obvious bug in it – we don’t really call the logger correctly!
When you think about it, who’s to say that inside the real virtual method the code actually calls the correct static method? No one. We could make our code call the “callLog” refactored method correctly, but we have no way to assure that the actual line inside the CallLog method actually calls the method correctly. That’s because we actually override that call in our tests. That’s a dangerous place to be and you need to watch out for making mistakes on this. Your interaction test might say that everything is ok, but you may forget to actually call the real methods correctly in the virtual calls.
This type of hazard does not exist with mock objects since with mock objects we don’t override any real implementation – just interfaces and method signatures.
This problem may not be as big as you think – I’ve rarely come across it. It’s kinda hard to find though, and it requires debugging to dig out. It would have been cought earlier with real Mock objects instead of using Mock Methods
Summary
Hopefully, you’ve seen that all good things have a dark side, but that doesn’t mean we can’t use them, we just need to know where the dark side begins and make sure we don’t go there (or that we at least know the way back). Mock objects and E&O(“close refactoring” as I like to call it) are two great routes to a testable application.
You’ll find the need to use E&O much more in legacy code than you would in new, green-field code, where you get the chance to write it testable from the beginning (Test-Driven code is often a great example of using interfaces and Mock Objects with the possibility of never needing to E&O anything).
The end result is all about being able to produce something out of these tests, and you’ll need to figure out which of these approaches works best for you.
I’d also recommend Michael Feathers’ great book entitled “Working Effectively with legacy code” for more methods and refactoring patterns to break code dependencies. Indeed, everything I’ve outlined here also appears there and may be considered an extension to some of the books ideas and thoughts.
You can also check out some of the Unit Testing Guidelines I keep adding to over time – there’s a whole category of them in my weblog.