What is a good example of unit test over specification?

I'm working on chapter 8 of my upcoming book and I need to come up with a good example of over specification in unit tests, that leads to an unmaintainable test in the long run (fragile test). I have my ideas on how to show it, but I'd love to know what you consider an over specified test, that can be shown with a concise (or even a bit contrived) code example.

- How would you define test over specification?

- how would you show it?

- are there multiple kinds of over specification with different ways to tackle them?

Published Saturday, March 01, 2008 4:08 PM by RoyOsherove

Comments

Saturday, March 01, 2008 4:58 PM by Bill

# re: What is a good example of unit test over specification?

I'd say over specification would be when you have a test that has more details of what is happening in the situation than a potential situation user must know is happening.

for example, if you had this method:

public int Add(int a, int b) {

   Logger.Log(LogLevel.Diagnostic, "Adding {0} and {1}", a, b);

   return a + b;

}

An overspecified test testing whether calling Add(3,5) would give the result of 8 and would mock ILogger and expect a call to Log. The user of this Add method should reasonably expect that the method adds 3 and 5 together, but should not expect that the method logs information at a diagnostic level.

Saturday, March 01, 2008 5:18 PM by Mark Seemann

# re: What is a good example of unit test over specification?

One of the most common examples of overspecified tests I've personally encountered is misuse of mocks where a stub (or similar) would suffice.

An (abstract) example would be a business component that depends on a data access layer. A unit test ought to verify that a business object is persisted properly by calling the correct method in the data access layer, but every other member invoked in the data access layer should be ignored.

However, in such a scenario, many test developers strictly define ALL expectations on the data access layer, instead of only the single, relevant method.

I must confess to having fallen into this pit myself back in around 2003-2004...

Saturday, March 01, 2008 10:47 PM by Michael Hedgpeth

# re: What is a good example of unit test over specification?

When the object you're testing doesn't have any real responsibility but passes things off to other objects, and so the whole test is a big mock expectation.  I start to think that my design is bad when I write these.  It's like I'm more worried about what's going on in the inside than the outside.

Code example:

class Validator : IValidator {

 public bool IsEntityValid(object entity) { }

}

class Repository : IRepository {

 public object AddToCurrentTransaction(object entity) { }

}

[TestMethod]

public void SaveWillValidateAndAddToRepository()

{

 // NMock

 Mockery mockery = new Mockery();

 IValidator validator = mockery.NewMock<IValidator>();

 Expect.Once.On(validator).Method("IsEntityValid"). Will(Return.Value(true));

 IRepository repository = mockery.NewMock<IRepository>();

 Expect.Once.On(repository).Method("AddToCurrentTransaction");

 Entity entity = new Entity(validator, repository);

 entity.Save();

 mockery.VerifyExpectationsHaveBeenMet();

}

The test is ok, but the problem is the entity here shouldn't have a save method.  It's not doing anything.  I'm overspecifying the exact steps it should take to overcompensate for a poor design.

}

Sunday, March 02, 2008 1:42 AM by Jack

# re: What is a good example of unit test over specification?

Phil Haack has a perfect example of over specification. Look at what is involved in testing a controller in the new ASP.NET MVC framework.

haacked.com/.../tdd-and-dependency-injection-with-asp.net-mvc.aspx

# The Disco Blog &raquo; Blog Archive &raquo; The weekly bag&#8211; March 7

Pingback from  The Disco Blog  &raquo; Blog Archive   &raquo; The weekly bag&#8211; March 7

Leave a Comment

(required) 
(required) 
(optional)
(required)