Is Typemock too powerful? Will it kill design for testability?

A couple of days ago, Typemock held a small "insiders" web meeting, showing off the new features of the next release. The presentation lasted for about 10 minutes, and we were using Webex to show live screen feed and chat was also enabled (similar to IRC)

Other than some nice feedback we got on what we showed, the most interesting part happened after the presentation was done, and we thought we'd give a few minutes for some small chat about various things.

Suddenly, a question was raised in the chat room about Typemock and good design (I'm paraphrasing): "What do you say to people who think that Typemock should be avoided because it's too powerful?"

There were about 4 Typemock employees in the chat , and 7 "insiders", and suddenly the flood of answers and new questions came in, without warning. In the chaos that ensued for about 15 minutes thereafter, one thing became clear to the powers that run Typemock: There is a big problem of FUD(Fear, uncertainty and doubt) about Typemock, that needs to be addressed.

What the FUD?

Many people are calling people who use mocking frameworks to avoid using Typemock because it makes it easy to avoid common design decisions that your tests "flag" as you try to write them. The claim is that Typemock , because of its ability to mock out practically anything, from statics, to LINQ queries to private constructors to sealed classes, makes the job of writing a test lose sight of design decisions about making the code testable in the first place.

That could be translated to "You will not design for testability if you use Typemock". But it can also be interpreted as :

  • I don't trust the people who write the tests enough to give them a tool that makes it easy to avoid design decisions.
  • People will default to a worst practice just because its easier
  • "Real" developers who write "real" tests will never need Typemock if they follow the TDD rules

 

So, let me try and address some of the concerns relating to design for testability and using Typemock:

1. You can still design for testability and use Typemock

It may sound weird, but just because you're using a tool that allows you to mock out static methods, doesn't mean you should go ahead and sprinkle statics as far as the eye can see. If your car lets you drive 200mph does that mean you'll choose to do that in a crowded street?

Test Driven Development has three main steps, and the first one is about writing a test that fails, even before the code under test is written. you can choose to write that piece of code under test in any design you like, and you can still do it with interfaces, Dependency injection and IoC principles.

For new, greenfield code, you can use TypeMock just like you could use Rhino.Mocks.  I know I've said other things in the past, but since I've been "inside" I've been using Typemock more and more and it really doesn't prevent you from doing anything you'd have done before.

2. If you were doing bad design before, the tool won't help you, just the technique.

Using TDD to get good design is still possible with TypeMock, especially if you use the Natural Mocks syntax (Record-replay, much like Rhino.Mocks). Choosing a framework based on it having more constraints is not a good idea. Why? Because you can still write poorly testable code with Rhino.Mocks if you weren't doing it TDD style.

3. Sometimes you just can't refactor your legacy code into testability.

Will you avoid testing it just because using non Typemock frameworks isn't possible? Or will you be pragmatic about it and do the thing that helps you write tests for code that is currently untestable? I would rather have a framework that gives me the choice of having a hammer and not using it, than not having it at all.

4. Pair Programming, Test reviews and Design reviews trump any tool

If you don't trust people to do good design, review the code with them.l use Agile techniques. If I write code with someone else, if I review tests that other people have written, and if I review designs other people came up with, I can tell on the spot what is poorly designed, what is not testable, not extensible etc.. If you're working alone and all you have is your framework to save you, you don't have much anyway. And it won't save you, it will just make it harder for you to write the code you want to write, with the design you want to use

5. If you're not sure how you'd design something and test it, you can always ask!

That's why god invented forums, mailing lists and the like. you can start at our new forum about techniques and best practices with unit testing. I'll be monitoring it personally and would love to start a conversation there or on this blog. whatever you like.

Published Thursday, January 17, 2008 7:23 PM by RoyOsherove
Filed under: ,

Comments

Thursday, January 17, 2008 8:03 PM by Haacked

# re: Is Typemock too powerful? Will it kill design for testability?

You know what I'd love to see you post is a code sample using the natural mocks syntax and one using the "full power" syntax.

It sounds like you're saying the programming model for writing tests using one approach vs the other in TypeMocks are different enough that you know when you're doing one thing or the other.

In other words, if you're writing new code, you won't "accidentally" be able to test something that's normally not testable by accident, thus perhaps cutting off an extensibility point. You have to consciously choose to use the more powerful approach for testing something like legacy code.

Is that a fair characterization?

Thursday, January 17, 2008 10:58 PM by Simon

# re: Is Typemock too powerful? Will it kill design for testability?

Old decision process

-Design some functionality that requires a single instance of something

-I will use a static class

-I want to test something that consumes the single instance

-Oh wait I can’t mock a static class

-I better use settable singleton or dependency injection through configuration.

Now it works but is more complicated because I need to make it testable

Decision process with TypeMock

-Design some functionality that requires a single instance of something

-I will use a static class

-I want to test something that consumes the single instance

-I will mock the static class using type mock

Now it works with no custom dependency injection framework or other code.

Do you understand what I am getting at? How can the logic be “don’t use TypeMock because that is not designing for testability”? TypeMock is facilitating that testability. If you can test something with TypeMock you are by definition designing for testability.

Now the only bad thing is that you are tying yourself to a testing tool. But that is the case no matter what tool you use. Each can facilitate you testing different things. Does “designing for testability” really mean “Design for testability if u only have the simplest toolset to help you”?

Thursday, January 17, 2008 11:35 PM by bnaya eshet

# re: Is Typemock too powerful? Will it kill design for testability?

In order to avoid the FUD problem you can use the following:

• Use different GUI presentation (icon, color, ext…) for test that may not apply to the TDD (and It can have different level from best practice to mal practice)

• Filter the test by their best to mal practice category so the tester can write any kind to test but the architect can validate upon the tests that will not bypass the TDD

Friday, January 18, 2008 3:44 AM by Casey

# re: Is Typemock too powerful? Will it kill design for testability?

I think everything you have said is correct, with a small proviso:

"If you were doing bad design before, the tool won't help you, just the technique"

The issue is with people just picking up TDD or mocking. They see that there is a way to mock a static class, and do it. The thought never occurs to them that it probably shouldn't be there in the first place. And because of that, when you try to then say "well I prefer to use an IoC container here to remove this dependency", they use the test as a justification of why you dont need to invert it, after all if we don't need to do it for testability, and our application probably won't have that logic replaced in future, why bother? We can just use the clever trick that TypeMock does if we *ever* see a need to replace it.

This pretty much comes from a paraphrase of a real example where I am working right now - where resistance to good design principles, and TDD, and mocking is pretty high. I think TypeMock would make some of my code *much easier* to test (where it depends on systems like MOSS), but for our own code I could see it being abused frequently.

It is that tricky balancing act to play - give people all the power in the world and trust them to use it safely, or don;t let them have it until they have a business (and technical) argument for using it.

Friday, January 18, 2008 5:34 AM by Joe Brinkman

# re: Is Typemock too powerful? Will it kill design for testability?

I love this quote "lose sight of design decisions about making the code testable in the first place".  To me this sums up what I perceive as a huge problem with many people in the current Agile/XP world: creating testable code is a means to an end, it is not the ultimate goal.  The easier it is to create testable code (because the testing tools are better), then I have more freedom to choose the best design to solve the business problem, instead of trying to find the best design to solve the testing problem.  The two concepts are not the same.

Friday, January 18, 2008 11:54 AM by Eric

# re: Is Typemock too powerful? Will it kill design for testability?

The real compelling argument for TypeMock is the legacy code argument.  When I starting a new project, I would probably pick Rhino.

For me, this is about using the right tool for the right job.  For new development, I find RhinoMocks to be the right tool.  But, I've done enough with legacy code to know that having the functionality that TypeMock provides is very useful when working against someone else's libraries that were obviously not written with testability in mind.

Sunday, January 20, 2008 6:00 PM by ISerializable - Roy Osherove's Blog

# The Case For Typemock

I just wrote an email to someone regarding Typemock . We were discussing why they don't think Typemock

Sunday, February 03, 2008 12:58 PM by Paulo Morgado

# Será Testar O Nosso Objectivo Final?

Na minha opinião, os habituais defensores de TDD equiparam-se a são fanáticos religiosos que pensam que

Tuesday, February 19, 2008 12:13 AM by Alec Lazarescu's Blog

# Testing Theories Part 4 - TypeMock Isolator - The Dark Horse of Unit Testing?

A lot has been discussed on the pro's/con's of IoC for the purposes of unit testing and in general it

Tuesday, March 04, 2008 7:46 AM by ploeh blog

# Typemock

Roy recently posted his thoughts on Typemock - whether it's too powerful and will kill TDD. His conclusion

Tuesday, March 04, 2008 7:51 AM by Noticias externas

# Typemock

Roy recently posted his thoughts on Typemock - whether it's too powerful and will kill TDD. His conclusion

Tuesday, March 04, 2008 8:47 AM by MSDN Blog Postings » Typemock

# MSDN Blog Postings » Typemock

Pingback from  MSDN Blog Postings  » Typemock

Wednesday, September 03, 2008 7:20 AM by Unit testing part 3: Design for testability « Freekshow

# Unit testing part 3: Design for testability « Freekshow

Pingback from  Unit testing part 3: Design for testability « Freekshow

# TDD in SharePoint - Using Dependency Injection | Agile Sharepoint development by 21apps and Andrew Woodward MVP

Pingback from  TDD in SharePoint - Using Dependency Injection | Agile Sharepoint development by 21apps and Andrew Woodward MVP