ISerializable - Roy Osherove's Blog

I picked up this piece of advice somewhere a year or so ago, and it's had a big impact on my testing.

There are two situations in which I use multiple asserts. The first is when additional information would be noise. For example:

Assert.IsNotNull(actualValue);
Assert.AreEqual(expectedValue, actualValue);

If I put these into separate methods, both tests fail (one assertion failure, one null reference exception). The null reference exception doesn't give me useful information beyond what I already know from the IsNotNull test. In fact, it's distracting noise. In this situation, I prefer to put the assertions in the same test method.

The other situation is when I'm testing a sequence of operations:

testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3
...

Even in this case, I prefer to test each step separately. But I'll test them together if, for example, I don't have a way to prep for doStep3() other than executing doStep1() and doStep2(). In other words, if I don't have enough control over the test subject that I can directly establish the entry conditions for doStep3().

I've started to consider that situation to be a design smell. Rather than splitting the tests, I now try to change the test subject to give me the direct control I want.

But even so, something tells me that I may want to test the sequence as above. The jury is still out on this one.

But in the vast majority of situations, my strong preference is to write one assertion per test.

In this case:

testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3

I'll usually do something like this:

public void testStep1()
{
testSubject.doStep1();
Assert something about step 1
}

public void testStep2()
{
testSubject.doStep1();
testSubject.doStep2();
Assert something about step 2
}

public void testStep3()
{
testSubject.doStep1();
testSubject.doStep2();
testSubject.doStep3();
Assert something about step 3
}

Not to say that there's not a design smell in there to uncover, but this gives you the separation of assertions you're looking for. In fact, it gives you very useful information about the sequence depending on what fails. I've even seen cases where testStep1 fails but the other 2 pass because something in the implementation of doStep2 righted the wrong imposed by doStep1.

Cheers,
David

I think you've mischaracterized the problem.

In

Assert.AreEqual(3, Sum(1001,1,2);
Assert.AreEqual(3, Sum(1,1001,2);
Assert.AreEqual(3, Sum(1,2,1001);

you don't just have multiple asserts. You also have multiple invocations on the function under test - and that's the flaw.

Compare that with, for example:

void test_ParseUserNameSimpleCase1() {
user = ParseUserName("John Q. Doe");
Assert.AreEqual(user.firstname, "John");
Assert.AreEqual(user.lastname, "Doe");
Assert.AreEqual(user.middlename, "Q.");
}

This has multiple asserts, all of which are really necessary for this particular case, but only one call on the function under test. Doing this as three separate test methods would just make it harder to read the tests and figure out what was being tested. If, for example, you left out the middlename assertion, it would be easier to spot this here than if the assertions were scattered. It's arguable whether separating them out would make it easier to diagnose any problems.

Also, there are times when it's justified to have multiple calls to the method under test, if it's stateful. But in such a case, it would be suspicious if the assertions verified anything other than the final state. In other words, I agree with David's example above, except that I would permit multiple assertions at the end, after all calls on testSubject.doStepX were done.

Finally, just because NUnit today stops at the first assertion failure doesn't mean that that's cast in stone. I see no reason that there couldn't, in theory, be AssertAndContinue.xxx, AssertAndAbort.xxx, and even AssertAndAbortFixture.xxx. In other words, the core problem isn't really putting multiple asserts into a test method; it's a limitation of the particular test framework.

Gary

I agree with this, you might consider changing "multiple asserts" to "multiple tests".

One other point, in my opinion this is OK:

testSubject.doStep1();
Assert something about step 1
testSubject.doStep2();
Assert something about step 2
testSubject.doStep3();
Assert something about step 3

However, your assertion should include a *good* message that describes what went wrong. That will direct you to the problem spot.

Roy Osherove has illustrated the need to Avoid Multiple Asserts in a test method. Basically i agree with

Roy Osherove has illustrated the need to Avoid Multiple Asserts in a test method. Basically i agree with

Roy Osherove has illustrated the need to Avoid Multiple Asserts in a test method. Basically i agree with