Announcing FsTest - A Testing DSL for F#
The Starting Point
During some of my research, I came across a now defunct project on Google Code called FsUnit. The idea behind this project was to put a syntactic wrapper around NUnit to make assertions a bit more readable and to use unique features to F# to make this happen. To be able to write such things as this for my assertions seemed powerful:
1 |> should (equal 1)
true |> should (be True)
And so on... Ultimately, I thought it could be improved and expanded, so I talked to the project owner, Ray Vernagus, and took some of the ideas and ran with them.
Looking at FsTest
I had a couple of issues with the code as it was. But, overall the ideas were quite sound. I also wanted to use xUnit.net as the back end testing framework as I'm pretty comfortable with it. The support for tests on static classes, and the ability to assert on throwing exceptions were some of the pluses that made me move in this direction. Listed below are some of the assertions that are supported:
Testing equality:
"foo" |> should equal "foo"
"foo" |> should not_equal "bar"
null |> should be Null
"foo" |> should be NonNull
"foobar" |> should contain "foo"
"foobar" |> should contain "hello"
Testing True/False:
true |> should be True
false |> should be False
Testing collections:
[1..10] |> should be NonEmpty
[] |> should be Empty
[1..10] |> should have 3
[1..10] |> should not_have 25
Exception Management:
throwsExceptionFunc |> should (throw_exception<ArgumentException>)
doesntThrowException |> should not_throw_exception
And some full examples might look like this:
#light
open System
open Xunit
open FsxUnit.Syntax
let throwsException() : unit =
raise(ArgumentException "Bad things!")
[<Fact>]
let throwsException_should_throw_exception() =
throwsException |> should (throw_exception<ArgumentException>)
[<Fact>]
let value_in_range_should_be_in_range() =
2 |> should be_in_range 0 100
It's a work in progress, but I feel that I have most of the assertion cases covered in this DSL. The ability through partially applied functions really shone when creating this library.
The Concepts Behind It
What makes this work syntactically is the use of the forward operator. I've covered this briefly in previous posts, but I'll elaborate on this again. This is one of the most important operators available to us in the F# language. This is the ability to invert the arguments in such a manner as the first argument is declared first and then the function statement is last.
This is how it's declared in F#:
let (|>) x f = f x
Which allows me to better express my intent and gives me the two advantages:
- Clarity of intent - allows you to perform various operations on the data in a forward chaining way such as:
let length = [1..10] |> List.map(fun x -> x * x * x) |> List.filter(fun x -> x % 3= 0) |> List.length
- Type inference - allows type information to flow from the left to the right and allows me to express my data first as part of my method. Shows better intention this way.
And similarly, something in C# would use extension methods to add a forward method, or maybe in this case a Should method which will then take a function as the second parameter, and then overload it as need be.
Using Actions might look something like this:
public static void Should<TArg1, TArg2>(this TArg1 arg1, Action<TArg1, TArg2> action, TArg2 arg2)
{
action(arg1, arg2);
}
And conversely to use the Func delegate might look like this:
public static TResult Should<TArg1, TArg2, TResult>(this TArg1 arg1, Func<TArg1, TArg2, TResult> func, TArg2 arg2)
{
return func(arg1, arg2);
}
public static void Contain<T>(IEnumerable<T> items, T item)
{
Assert.Contains(item, items);
}
And to use it would look something like this:
Enumerable.Range(1, 10).Should(Contain, 3);
But, I'm not necessarily a fan of that style of syntax. C# isn't quite a functional language which allows me to do such clever things with operators. I don't think it will ever gain that level of expresiveness required.
Instead, I created two functions that could be chained together such as the should function and the be function. This allows me to create some of the examples above. Let's take a look at each function:
let should f actual =
f actual
let be (f : 'a -> unit) actual =
f actual
So, that will allow me to express such things as:
true |> should be True
And the true function might look something like this:
let True condition =
Assert.True(condition)
As you can see, there isn't much to this, and covers most of the assertion scenarios that I've encountered.
Where To Go From Here?
So, where am I headed with this? For right now, I'm happy with the assertion syntax I've been able to achieve with the language. For the future, I'm looking at creating a more expressive way for doing BDD and the expresiveness of F# I think can handle such things. I've been following the Specs project for a little bit and I think they have a few good things going for them, although Scala isn't a functional language in the way that F# is, so I'm not sure how many lessons can be applied. Specter is another interesting one as well as Machine Specifications (MSpec) from the one and only Aaron Jensen.
I'd like to thank the F# team, Dustin Campbell and Harry Pierson (DevHawk) for their help on this as well. The team around F# is a wonderful team and I can't wait for the CTP of F# to ship. Makes me excited sometimes when working for Microsoft like I do.
But, in the mean time, feedback is appreciated. Tell me what works and what doesn't.
