"Isolation" seems like a good idea. Did you think about just "Double"? Still too overused, or too specific?
Interesting notes, Roy. Lots of good lessons learned in there.
This one in particular, though, sounds a little dubious... like marketing material that is only telling half the story:
"Isolator was the only one where people didn’t ask me a single question about what API to use. they did put calls in the wrong place in the test though (AAA is hard to get used to)"
What order are you teaching the frameworks in? If you're teaching Isolator last, then you are stacking the deck in favor of the Isolator syntax because everyone will already be familiar with other mocking framework syntax and APIs. Since all mocking framework APIs are well within the same realm of language, it's relatively easy to switch between them and understand what's going on.
if you are teaching Isolator last, then that claim in irresponsible at best. You would need to change up the order in which you teach the frameworks, across many different groups of students, to get an accurate picture of which syntax is picked up easier.
Derick: Interesting. I did show Isolator last. But for the opposite reason. I was afraid that showing it first would give it some unfair advantage because people tend to remember the first API they learn (Rhino was first).
Next time I'll teach it first and see what happens.
BTW: the api difference between rhino, Mow and Isolator is so big I don't think that was the reason. I really feel it's just that much simpler (single point of entry was the main reason)
Thanks for sharing! Very interresting.
> i think Isolator is the only one that will re throw swallowed exceptions).
Rhino Mocks will rethrow exceptions as well
Excellent notes, Roy. This should help folks designing unit testing and mocking frameworks.
Few things I wanted to comment on. You said,
The word “Expect” confuses because it makes you think about “expectations” as in interaction testing, when in fact it can be used for stubbing out things.
After using RhinoMocks for about a year and a half now, I tend to agree. There is a call in Rhino that makes a distiction between stubbing out things and expecting a call:
// Stubbing out a value:
SetupResult.For(foo.Bar).Return(42);
// Setting up an exptectation:
Expect.Cal(() => baz.Shabam());
We tend to use this distinction wherever we can. Helps clarify intent of the unit test.
Also, you mentioned that unit test names really help flesh-out the spec. I now name all my unit tests something like:
[Test]
public void ClickingFobnicatorLaunchesNewBuzzTap() { ... }
I've also found this to be useful for maintenance purposes -- if you break some code, your unit test will fail -- you can hop right to the unit test and find that the code you broke is the launching of new buzz taps when clicking a fobnicator.
One last thing: I have tried to move to AAA syntax. Honestly, I find record/replay cleaner and more structured!
Very interesting notes, and thank's for sharing. I have to tell you that you're one of the VERY few that has not overlooked VB.Net for using mocking frameworks. Ofcourse the next version of VB is going to support Lambda-statements as well as Lambda-expressions so that will help a lot. I really like what you were doing for the VB-friendly API for Typemock, it looked really cool.
Quite interesting to read the reactions of the course participants, I do recognize many of the reactions from my own experiences. I was just interested as to why you or the participants feel that TestDriven.NET is soo much better than resharper as a unit test runner?
Only TypeMock can be considered an "isolator framework". The other frameworks are just mock frameworks. The isolation is coming out of explicit design considerations. The source of isolation is design, not tooling.
Urs:
Strictness means that you could only create strict mocks, not nonstrict (unexpected calls throw an exception). we found no way to turn it off (perhaps there is one in the latest version?)
Regarding exception swallowing: If NMock throws a verification exception, and the code in production has a try catch on all exceptions, the NMock exception will also be swallowed.
Thanks for the clarification.
Currently there is no direct way for nonstrict mocks. Only way would be to define a Stub for all Methods and all Properties and so on: Stub.On(mock).Method(new AlwaysMatcher()); But it should be an easy extension to allow stubbing of whole interfaces.
Exception swallowing: we could rethrow a swallowed exception in the VerifyAllExpectationsHaveBeenMet method.
We'll think about that.
And thanks again for the very useful feedback!
Just wanted to stop by to say that the next version of NMock2 will
- rethrow swallowed exceptions in the Verify.. method
- be able to mock classes (virtual and abstract methods)
- stubs are supported (no need for expectations for simple stubbing)