Future of Mock Frameworks: AEIS - Abstract Expectations & Isolation Syntax
As I was looking back and forward on the changing syntax of testing and mock frameworks, it occurs to me that there are still things that need to be solved in the mocking space. many things.
- It's hard to choose a framework and stick with it because they have different abilities which you are not sure you might need or not later on
- Each framework has different syntax which is a learning curve and maintainable curve in test code
- You can't change your mind easily after you choose
- There is no specified set of known features to be expected. Each one has what it thinks is best.
What if we had an abstract layer on top of our mocking frameworks? Just like ADO.NET is an abstraction layer, and the Data Application block is an abstraction layer, NHibernate is an abstraction layer - Mocking frameworks are infrastructure libraries that, if looked at well enough, are pretty much trying to do the same things in different ways (like databases)
Here are the main services a mocking framework can provide(if I miss something tell me):
o Stubs
§ Create Canned Answers
o Mocks
§ Create Expectations
· Record-Replay Model
· Explicit Model
· Parameter Constraints
· Custom param constraints
§ Partially Implemented Mocks
§ Pass-through mocks
§ Strict\non strict
§ IoC- Auto mocking containers
o Mocking Abilities
§ Strict (Traditional Interface based)
§ Non Strict (Typemock,JMockit Stuff)
There are two main services here: Expectation abilities, and Creation\Isolation abilities.
what if there was a framework that would allow you to write code like this:
[Test]
public void CreateSimpleInterfaceBasedMock()
{
MockingEngine.Provider = new RhinoMocksProvider();
IMock<ILogger> mockLogger =
Represent.Type<ILogger>().AsMock();
...
}
What would the proposed AEIS (Abstract Expectation and Isolation Syntax) look like?
I would consist of a very thin API wrapper that allowed all the different services mocking frameworks do today, but it would not do this functionality on its own. instead it would use a provider model that will allow one to write adapters for specific mocking frameworks, forwarding calls and translating the different syntaxes between AEIS and the specific framework. I will also have a rules engine (on the left) that will allow creating custom rules that will run when using various APIs (perhaps using an event driven relay) that will allow to enforce specific usage of specific features.
what would be some of the benefits in something like this?
You can change mocking frameworks without changing lots of code
There is an investment to be made when choosing a mocking framework today: You spend time learning the syntax, you write tests with that syntax and you plan on not learning a new syntax to do slightly different things. A year later mock framework X comes in and you think that you'd like to use it, but you've already invested so much time and money in the original one that you aren't going to try. You lose valuable features just to avoid the risks.
You can add rules “on top” of the mocking framework
An abstraction layer can have custom code that will allow you to write custom logic that makes sure developers are using the framework in a way that you'd like. for example, you'd like to use Typemock Isolator but you are afraid that people will abuse the ability to write tests for code that isn't "testable" by default (no interfaces etc..)
I call that"Strict" mocking policy vs. "loose" mocking policy (or "classic\traditional vs. modern..."). So the
o Allow only strict mocking syntax, for example
One syntax to remember and learn for all types of jobs
Simplify and overcome problematic mocking syntax in existing frameworks
And there are definitely some around. For example, Typemock Isolator currently has a very confusing Mock() and MockObject() set of methods which sometimes I personally forget what each one does. Sure, we'll change it, but RhinoMocks also has some confusing syntax (PartialMocks() vs. DynamicMock() Vs. Mock()...)
Having a top level API can help alleviate and standardize some of these issues.
Provide a recipe for mock framework builders
If you have to build a provider for your mocking framework, you have to have a contract of what you can and cannot provide. this will also help standardize the essential set of services that mocking frameworks provide today or at least, the set of things you need to consider when choosing among them as the underlying engine.
It will help avoid religious wars (like design for testability) by allowing you to choose easily and enforce to your standards through the common API.
What about testing frameworks? why not start there?
I think currently most test frameworks (XUnit.net aside) are pretty much interchangeable (Nunit and MbUnit and msTest) so there is less pain there. However, take a look at the next version of MbUnit , running on the Gallio automation platform) which tries to provide a generic test runner model for all test frameworks in .net. I think they are trying to solve the same set of pains and the same set of benefits.