So what if my method-under-test is responsible for coordinating and setting up multiple things? How do you verify that each thing has been set up properly by this method without doing multiple asserts?
Oran: I'd advise a separate test for each check, each one calling the method on an preinitialized object and then doing one assert.
That way each test can also be named appropriately.
Roy,
While I completely agree with you in theory, I find it very time consuming to do that and I feel that the benefit isn't always warranted.
For example, say you create a simple class and you write some tests for it.
I'd start of with making sure that the constructor satisfied the invariants (like:
User u = new User("Fred", "Flinstone");
Assert.AreEqual("Fred", u.FirstName, "First name wasn't set");
Assert.AreEqual("Flinstone", u.LastName, "Last name wasn't set");
in this example I do 2 asserts, but the test is really CanCreateUserWithInvariants(). The message can tell me which assert failed. Is this wrong? would you really separate this into 2 tests?
My 2nd comment (really a question) is this:
I find myself wanting to create DEPENDENT tests, and before you shudder think of this: Say you have some functionality that you build upon in your tests. Your model grows and each addition is well tested. Now you want to test higher level abstractions, so these tests now DEPEND on the functionality of other tests, like making sure our User class can actually set its own properties.
If that part fails now I have 2 (or likely a lot more) failing tests. If NUnit had an attribute or something that specified DependsOn("CanCreateUserWithInvariants") then the GUI can mark those as Purple or something to say that other tests must be passing before this one runs.
I realize this opens the door for serious abuse of dependent testing, but maybe you could shed some light on it for me or recommend a better way of testing :) How can I ensure that my tests are atomic and singular, but also leverage the code I've written to provide even more robust tests?
Hi Roy, great post as per usual.
I have recently trended towards smaller, more targetted unit tests myself. Whilst I find that this makes tests easier to read and failed tests easier to rectify, I find that writing them is more burdensome.
When you get a chance, would you mind blogging a bit about alternatives to implementing unit tests in C# / VB / whatever. Are there any unit test-specific CLR languages out there that would make unit tests easier to write, read and maintain?
Thanks,
Kent
I do not agree with the part of having to know which one of the asserts failed. As long as the whole test fails it fails. Sure it would be easier to find the concrete assert just by looking at the code, but ... I preffer less test code overall than less code in one method, but that is my opinion.
And for your second argument. If you expect your method to run differently after each call you should also test that it does it in a correct, intended way and write your tests with this in mind.
Hi Ray.
When I'm using UT, I usually have more than one assert per test.
Why? Simple: when an assert is successful you have a perfect consistency of the object state and using messages you can easily determine what was going wrong.
Plus, sometimes you need to test more "aspects" of a feature and doing several tests instead of one covering more aspects is dispersive in the task switching.
That said, I'm not advocating one test method per class, rather I'm advocating one test method per logical block inside the class.
For example: if I have a class that renders an ul HTML tag and various li's and then I add a property to the li class like "useAnchor" and an "AnchorUrl" one, then it's likely I'll want to test those in the li tag generation rather than in another test which would inherently be burdensome to write.
Cheers,
Andrew
I think the main reason (that the first assert fails and the others don't run) is a valid reason to have single assert per test, but I think if that was mitigated it would largely become a matter of preference.
You mention that you would extract the setup code to a private method, but you now run this on every test. So if you setup code creates 5 new records in the DB, you know have to do that 10 times instead of one. This is a problem we have on our current project, we have so many tests and so much startup code that it takes forever to run all of our unit tests.
I tried to think of an instance of where I have written an assert that changes the state of an object and I couldnt think of one... could you provide some examples?
-James
Once again, you hit the nail right on the head, Roy. Nice job.
How are things going, by the way? Drop me a line when you have a chance....
I hate it when I get a Nullexcetion with Assert.IsTrue(addressList[0] != null,) because my list is null, I've much rather have an Assert.IsNotNull(addressList) fail on the line before, to give me a better idea what went wrong. I realize the idea is you would have 2 test, and they both would fail, but the problem is one would fail because of the assert, and the other would fail because of a null exception, thus hiding the real error.