Throw-Away Tests Vs. Tests that Last - ISerializable - Roy Osherove's Blog

Throw-Away Tests Vs. Tests that Last

TATs  - That's what I call the tests people write when they are writing unit tests for the first time on a real project.

image Those tests usually don't last more that a few months before they either get deleted or ignored completely because it takes too much time to maintain them, or no one understands what they are trying to test, or no one understands why they keep failing when the code seems fine, or all of the above combined.

What's the difference between TATs and TTLs (Tests That Last)?

 

TTLs are:

  • Maintainable - Maintaining your production code when you have unit tests should not take much longer than maintaining it if you didn't have tests. for example - Assume a class that has 50 tests and suddenly you decide to change the constructor signature for that class. how long would it take you to make sure all the 50 tests for that class run again (or even compile)?
  • Readable - Can you understand the test that somebody else wrote? Do you have to debug it to understand what it's testing? can you tell the intention just by looking at the test name? if it failed, will you know where the problem is?
  • Trust-worthy - When you finish writing a feature, how comfortable do you feel just running all the tests and seeing them all pass? Does your test test the right thing? Do you have tests that keep failing but you think it's a normal behavior that they fail?  Do you have tests that seem to only pass if run in a specific order? Do you have tests that run other tests?

Take any one of these away and the others crumble as quickly as I do when my wife asks me to take the garbage out. As a bonus I can tell you that you  what you have if even one of these is true is a TAT, not a TTL.

Have you ever written TATs?  Were you able to make the move to TTL? If not, why?

Published Thursday, September 13, 2007 7:01 PM by RoyOsherove
Filed under: , , ,

Comments

Thursday, September 13, 2007 9:31 PM by Peter Hancock

# re: Throw-Away Tests Vs. Tests that Last

Plenty of throwaway tests. It took me a long time to make the transition.  I think its because I was still thinking design first then test rather than thinking about writing the test first and using that to drive design.  Usually starts with...

"OK - I need a GST Calculator object to test to ensure that 10% of $1,000 is $100" - rather than...

"OK - if I purchase a bike at $1,000, I have $100 in GST that needs to be allocated"

The former tends to lead to tightly coupled tests - something I blogged about recently at

www.bottleit.com.au/.../Loosely-couple-your-tests-to-your-implementation.aspx

The other barrier I notice is people failing to apply refactoring principles as rigourously to tests as to "production" code.  When you promote test code to be a first class citizen along with "production" cod, I find you're on the path to rightousnss and TTL.

Friday, September 14, 2007 3:24 AM by Avish

# re: Throw-Away Tests Vs. Tests that Last

This is the first time we've been using unit tests, and yes, most of them are TATs. Since it's the first time, most developers are still skeptical about unit tests and their benefits. Test code is way beneath a first-class citizen; probably a third class-citizen, with production code and developer tool code taking up the first two places. Testing First is of course unspeakable at this point.

However, at least some of our tests are TTLs; incidentally, those are the unit tests that are closer to being integration tests, since in that level it is a lot easier to make the test show the intention of the code it's testing.

I expect that that with time, a better understanding of testing will grow on us and we'll start making more TTLs. Wish us luck!

Friday, September 14, 2007 12:03 PM by Thomas Eyde

# re: Throw-Away Tests Vs. Tests that Last

My last project was full of them! Today, the test project is abandoned due to this.

The main reason, I think, was badly written interaction based tests. I guess no one, except me, understood them. They were also tightly coupled to implementation, so whenever the implementation changed, the tests broke.

Burned by this, I avoid mocking frameworks, as they lead to tests that "no one understands what they are trying to test".

Friday, September 14, 2007 6:30 PM by Dave Diehl

# re: Throw-Away Tests Vs. Tests that Last

Definitely written more throw away tests than tests that last.  

I completely agree that not refactoring test code and not applying good design principles leads to throwaway tests.

One strategy that has been suggested to me is to use LONG test names to make them descriptive.  For example, a name  like "CanCalculateCostOfBikeWhenStateTaxApplied" would be fine for a test, but too long for normal code.  It will be interesting to see how effective this strategy is.

Saturday, September 15, 2007 8:27 AM by Jeff Brown

# re: Throw-Away Tests Vs. Tests that Last

You sure got me thinking!

I stayed up until 5:30am writing up my thoughts: blog.bits-in-motion.com/.../tests-that-last.html

Bed.  Now.  ;-)

Saturday, September 15, 2007 12:39 PM by Joe Ocampo

# re: Throw-Away Tests Vs. Tests that Last

I would say that TAT is the direct result of forgetting the refactor part of "Red, Green, Refactor" meaning you not only refactor your code under test but you also refactor your test to a more soluble state.

Monday, September 17, 2007 12:47 AM by Peter Hancock

# re: Throw-Away Tests Vs. Tests that Last

@Dave

Long names are a great strategy.  I've used them in the past to highlight ridiculously long methods of legacy code...  along the lines of

CalculateFeeForTransactionAndApplyFeeToAccountThenReportFeeToCustomerByEmailAndNotifyAccountsDepartment

The first time I did that I got hauled over the coals for a ridiculously long method name.  I raised the suggestion that perhaps it was a ridiculously long method and that they were shooting the messenger.  

Needless to say, it raised the issue and highlighted some pain points in the code.  It's now a technique I use in my own teams.  And it's surprisingly effective.

Tuesday, September 18, 2007 3:09 AM by Pierre-Emmanuel Dautreppe

# XP : Simple Code vs Simple Solution

XP : Simple Code vs Simple Solution